0

Is it possible to configure AWS CloudWatch with a filter/monitor that "listens" or watches for a particular type of log message (ideally with granular or regex-like control where I can tell CloudWatch to look for a particular pattern in the log message) so that it forwards the log message off to a particular SNS endpoint?

Meaning:

  1. My app publishes log messages to CloudWatch
  2. CloudWatch is configured with this filter/monitor to listen for log messages matching a particular regex/pattern
  3. Any messages matching this pattern get forwarded on to an SNS endpoint of my choosing

The best I could find was this article which shows how to have CloudWatch send email through SNS, but not sure if the alarm they use can be configured to watch for message patterns, and not sure if SNS can be configured to do non-SES/email related downstream work.

hotmeatballsoup
  • 385
  • 6
  • 58
  • 136

1 Answers1

1

CloudWatch logs can have subscriptions. The targets can currently be setup for Kinesis streams or Lambda functions, but you could define the subscription filter to send matching messages to a lambda function that puts them onto the SNS topic, if that is required.

For example:

aws logs put-subscription-filter --log-group-name /aws/ecs/mycontainer --destination-arn arn:aws:lambda:us-east-1:123456:function:my-log-watch-sns-feeder --filter-name container-errors --filter-pattern "ERROR"

This would setup a subscription filter that sends log messages from an ECS container called mycontainer that contain the string ERROR to the lambda function named my-log-watch-sns-feeder.

For more information:

Tres' Bailey
  • 709
  • 7
  • 17
  • Thanks @Tres (+1) but what about the article I posted? There they demonstrate how to configure a CW alarm that sends messages to SNS directly (not through Kinesis Streams or lambdas)...thoughts? Thanks again! – hotmeatballsoup Oct 05 '18 at 16:14
  • I could be wrong, but I think the only way to get it to work would be to create metrics from your logs, as described in https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CountOccurrencesExample.html then create your events and alarms from there. Regardless, it is definitely possible to have other subscribers than just SES/email. – Tres' Bailey Oct 05 '18 at 16:50