I want to query CloudWatch metrics for all EC2 instances in my account (1000+) that reported them during a specific 1 hour time frame (load test) using Boto3. It is expected that instances would have been brought up/down by auto-scaling during the test.
I've tried using search expression as suggested in this StackOverflow answer for querying RDS instances, but this is limited to 500 results with no way to paginate according to the CloudWatch search expression syntax docs:
Search expression limits
The maximum search expression query size is 1024 characters. You can have as many as 100 search expressions on one graph. A graph can display as many as 500 time series.
I could do multiple queries for metrics by specifying instance ids using something like this:
for instance_id in instance_ids:
response = boto3.client("cloudwatch").get_metric_data(
MetricDataQueries=[
{
'Id': "network_out",
'MetricStat': {
'Metric': {
'Namespace':
"AWS/EC2",
'MetricName':
'NetworkOut',
'Dimensions': [{
'Name': 'InstanceId',
'Value': instance_id
}]
},
'Period': 300,
'Stat': 'Sum',
'Unit': 'Bytes'
}
'ReturnData': True
}
]
)
but I would still need to have the full list of instance_ids
that reported metrics during the time frame, including those which were terminated by auto-scaling.
How can I query CloudWatch metrics for all EC2 instances which reported during the time frame?
A way to get a list of all EC2 instances which reported metrics during that time frame would also work.