How to get AWS VCPU's utilization number via API

Question

I would like to get the number of AWS VCPU's used by all running instances in the region via GO API, so I can run some preflight checks before I create new instances, to see if there are enough VCPU's available, without increasing the quota.

In other words, if we look at the console we can see that Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances service quota, shows Applied quota value column that indicates the current VCPU's limit. If you click on this field, you can see the 'Utilization' field that shows the used VCPU's. This is what I need.

AWS Service Quotas - Running On-Demand Standard instances

Thanks!

Do you have perhaps a screenshot of what you are after? Your question is not very clear. — Marcin, Mar 26 '22 at 23:29

score 0 · Answer 1 · answered Mar 27 '22 at 00:48

To discover where this information was available, I looked into the Service Quotas AWS service.

The aws service-quotas list-service command lists the service codes, such as:

{
    "Services": [
        {
            "ServiceCode": "AWSCloudMap",
            "ServiceName": "AWS Cloud Map"
        },
        ...
        {
            "ServiceCode": "ec2",
            "ServiceName": "Amazon Elastic Compute Cloud (Amazon EC2)"
        },
        ...
        {
            "ServiceCode": "xray",
            "ServiceName": "AWS X-Ray"
        }
    ]
}

So, the service code for Amazon EC2 is ec2.

Next, I listed the quotas for this service using aws service-quotas list-service-quotas --service-code ec2. Amongst the output I found:

{
    "Quotas": [
    ...
        {
            "ServiceCode": "ec2",
            "ServiceName": "Amazon Elastic Compute Cloud (Amazon EC2)",
            "QuotaArn": "arn:aws:servicequotas:ap-southeast-2:123456789012:ec2/L-1216C47A",
            "QuotaCode": "L-1216C47A",
            "QuotaName": "Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances",
            "Value": 640.0,
            "Unit": "None",
            "Adjustable": true,
            "GlobalQuota": false,
            "UsageMetric": {
                "MetricNamespace": "AWS/Usage",
                "MetricName": "ResourceCount",
                "MetricDimensions": {
                    "Class": "Standard/OnDemand",
                    "Resource": "vCPU",
                    "Service": "EC2",
                    "Type": "Resource"
                },
                "MetricStatisticRecommendation": "Maximum"
            }
        },

    ...
    ]
}

This seemed to match the quota displayed in the Management Console.

I then noticed that the UsageMetric information looked like an Amazon CloudWatch metric:

            "UsageMetric": {
                "MetricNamespace": "AWS/Usage",
                "MetricName": "ResourceCount",
                "MetricDimensions": {
                    "Class": "Standard/OnDemand",
                    "Resource": "vCPU",
                    "Service": "EC2",
                    "Type": "Resource"
                },
                "MetricStatisticRecommendation": "Maximum"
            }

I then went to Amazon CloudWatch, clicked All metrics and entered a search term of usage standard:

Amazon CloudWatch metrics search

Aha! The Standard/OnDemand metric appeared!

Selecting the checkbox then showed a chart:

Amazon CloudWatch metrics chart

This metric matches the Utilization figure shown in the Service Quotas page.

Bottom line

Service Quota metrics are available through Amazon CloudWatch. You can use standard API calls to retrieve these metrics, or access them through the Amazon CloudWatch Metrics management console. You can also Create Alarms on these metrics to notify you when they exceed desired limits.

Thank you so much for your detailed answer John. I found this: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/viewing_metrics_with_cloudwatch.html and the value I need is the 'ResourceCount'. I tried to use 'awsClient.cloudwatch.GetMetricData()' but that won't help. If you can point me to the api call to fetches the ResourceCount by passing the service name, type,resource class as mentioned in the link above, that would be great. — Tzvika Avni, Mar 27 '22 at 11:51
This is a better one: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/viewing_metrics_with_cloudwatch.html#service-quota-metrics — Tzvika Avni, Mar 27 '22 at 11:57
So I can do this in my code: listMetricsInput := cloudwatch.ListMetricsInput{ Dimensions: dimentsions, MetricName: &metricName} ; metricsData, err := client.cloudwatchClient.ListMetrics(ctx, &listMetricsInput) where the dimensions include the right input. This find my metric !! But I still do not see how do I get the ResourceCount. — Tzvika Avni, Mar 27 '22 at 14:10
I also tried MetricDataQuery but did not get values in the response. — Tzvika Avni, Mar 27 '22 at 16:12
I was able to retrieve the metrics with: `aws cloudwatch get-metric-statistics --metric-name ResourceCount --start-time 2022-03-20T00:00:00 --end-time 2022-03-25T00:00:00 --period 86400 --namespace AWS/Usage --statistics Maximum --dimensions Name=Service,Value=EC2 Name=Type,Value=Resource Name=Resource,Value=vCPU Name=Class,Value=Standard/OnDemand` (I found that it required _all_ of those Dimensions to return a result.) — John Rotenstein, Mar 28 '22 at 01:33
You are the best !! Thank you so much. One last question - Since I need the current maximum resource count, I set the endTime to Now(), and the startTime to Now() - 5 minutes. If I set the the startTime less then 5 minutes, I do not get the result. Is that expected? It does not let me set the startTime to Now() of course. It returns error. — Tzvika Avni, Mar 28 '22 at 06:51
The metrics probably only update every hour or so. Just set a longer time period and always use `Maximum` to discover the biggest value it was recently. — John Rotenstein, Mar 28 '22 at 11:50

How to get AWS VCPU's utilization number via API

1 Answers1