AWS Custom CloudWatch metrics - Aggregate by Auto-Scaling group

Question

I'm trying to set some custom AWS CloudWatch metrics using the Java SDK.

I can't seem to find anything in the documentation describing how to get certain pieces of data, nor what data I need to include.

MetricDatum datum = new MetricDatum()
    .withDimensions(
        new Dimension()
            .withName("InstanceType").withValue(/* 1 */),
        new Dimension()
            .withName("InstanceId").withValue(/* 2 */)
        /* 3 */
    .withMetricName("My metric").withTimestamp(new Date())
    .withUnit("Percent").withValue(new Double(55.0));

So, questions (for each of the commented numbers in the code above):

1. Where do I get the data to put here, using the Java AWS SDK?
2. Where do I get the data to put here, using the Java AWS SDK?
3. What other data do I need to include in order to ensure I can aggregate by auto-scaling group? (aggregating by security group would also be fine)

For #1, I've seen that I can make a regular HTTP call to http://169.254.169.254/latest/meta-data/instance-id to get the instance-id, but I'm hoping to do this all via the AWS SDK, if there are methods available to do so.

Answer 1

I posted the question to the Amazon support team.

The EC2 documentation gives a list of URLs that can be called to grab a bunch of meta-data, including the InstanceType (question 1), the InstanceId (question 2), and the security group (question 3).

The auto-scaling group can be obtained using the AWS SDK for Java, by getting a list of all the auto-scaling groups, and then iterating through that list until you find the instance with your own instanceId (that was retrieved using the URL listed above):

String instanceId = "Your-InstanceId";
AmazonAutoScalingClient amazonAutoScalingClient = new AmazonAutoScalingClient(new BasicAWSCredentials(accessKey, secretKey));
DescribeAutoScalingGroupsResult describeAutoScalingGroupsResult = amazonAutoScalingClient.describeAutoScalingGroups();
for(AutoScalingGroup autoScalingGroup : describeAutoScalingGroupsResult.getAutoScalingGroups()) {
    for(Instance instance : autoScalingGroup.getInstances()) {
        if(instance.getInstanceId().equals(instanceId)) {
            return autoScalingGroup.getAutoScalingGroupName();
        }
    }
}

Answer 2

It's not in the javadocs, but there's a handy utility class called EC2MetadataUtils that'll give you metadata info such as InstanceType (1) and InstanceId (2).

As far as getting the ASG name, AWS documents that

when you launch an instance in an Auto Scaling group, Auto Scaling adds a tag to the instance with a key of aws:autoscaling:groupName and a value of the name of the Auto Scaling group

So you can save a little looping by just fetching the tags for the instance

String instanceId = EC2MetadataUtils.getInstanceId();

String asgName = null;
List<TagDescription> tagDescriptions = new AmazonEC2Client().describeTags(
        new DescribeTagsRequest().withFilters(
            new Filter().withName("resource-id").withValues(instanceId)
        )
).getTags();
for (TagDescription tagDescription : tagDescriptions) {
    if ("aws:autoscaling:groupName".equals(tagDescription.getKey())) {
        asgName = tagDescription.getValue();
        break;
    }
}

The Dimension name you would use to ensure you can aggregate by Auto Scaling Group Name is AutoScalingGroupName (3)

new Dimension().withName("AutoScalingGroupName").withValue(asgName)

Answer 3

Trying to implement this same thing with version 1.10.17 of the AWS Java SDK, and the solution provided by @Tinclon in the accepted answer only returns null for the auto scaling group. The following snippet, however, did correctly return the auto scaling group for me.

    String getAutoscalingGroup(final String instanceId) {
    final DescribeAutoScalingInstancesRequest describeRequest = new DescribeAutoScalingInstancesRequest().withInstanceIds(Collections.singleton(instanceId));
    final DescribeAutoScalingInstancesResult result = autoScalingClient.describeAutoScalingInstances(describeRequest);

    for (AutoScalingInstanceDetails details : result.getAutoScalingInstances()) {
        if (StringUtils.equals(instanceId, details.getInstanceId())) {
            return details.getAutoScalingGroupName();
        }
    }

    return null;
}

I didn't try this with other versions of the SDK, but the functionality is equivalent with the functionality in the (as of today) latest version of the AWS command line client

Answer 4

You can aggregate by image id:

curl http://169.254.169.254/latest/meta-data/ami-id

Typically, an auto-scale group runs on a dedicated image, so this approach should work.