Create an alarm based on a CloudWatch insight query

Question

My problem:

I would like to blacklist IPs which are accessing my public AWS API Gateway endpoint more than 5 times a hour.

My proposed solution:

Requests are logged to CloudWatch
Requests are counted and grouped by IP
An alarm monitors IPs send a message to a SNS topic in case the threshold is met
Lambda is triggered by the message and blacklists the IP

I am able to log and count the IPs by using the Insight query below:

fields ip
  | stats count() as ipCount by ip
  | filter ispresent(ip)
  | sort ipCount desc

What I am struggling to accomplish is getting an CloudWatch Alarm based on this query.

I have searched a lot but no success. Any ideas on how to create such a metric / alert?

Can you try with this approach "put IP/count in dynamoBD and run another lambda to trigger SNS every 5 minutes if match condition"? — Franxi Hidro, Jan 23 '22 at 10:11
How big is your IP/Count data? million rows or thousand rows — Franxi Hidro, Jan 25 '22 at 10:35
@FranxiHidro Are you planning to run the insights query every X minutes or something similar? — Kaustubh Khavnekar, Jan 25 '22 at 11:27
I just want to restrict users to access it more than 5x / hour keeping the API Public — Kaguei Nakueka, Jan 25 '22 at 12:22
@KaustubhKhavnekar Yes, that is my idea, a cronjob to check if count >5, then (do something) block that IP. KagueiNakueka, you may need a WAF for your public API to do that feature, and I think my solution is cheaper. — Franxi Hidro, Jan 25 '22 at 18:30

Register Sole · Answer 1 · 2022-01-28T03:51:58.213

I know you planned to do a custom Lambda, but check if WAF already fulfills your use case. For example, the rate limit section in this article here clearly allows you to define the rate per 5-minutes for a given IP:

https://docs.aws.amazon.com/waf/latest/developerguide/classic-web-acl-rules-creating.html

If you are not doing anything else, a custom Lambda function may not be needed.

EDIT

If you want to go down the path of CloudWatch alarms, I think you can define a metric filter to create a CloudWatch metric. Then you can create the alarm based on the metric.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringLogData.html

score 0 · Answer 2 · answered Feb 01 '22 at 07:35

The best approche is to use the managed services "AWS WAF" which is perfectly integrated with your APIs.

The problem with a custom solution, is the latency, time to aggregate logs, count, and the cost, because each time a lambda will run with queries....

In API Gateway you can attach a WAF Web ACL directly, you can indicate the rate per 5 min, per 10min... for you need, is the job of the WAF.

Create an alarm based on a CloudWatch insight query

2 Answers2

EDIT