How to verify the status of an EC2 instance launched from a newly created AMI?

Question

How to check if an EC2 instance is up and running/one particular service is up or down using boto3. The instance is created from a newly created AMI.

Answer 1

You can uses the following Instance:

ec2 = boto3.resource('ec2')

instance = ec2.Instance('instance_id')

print(instance.image_id)

Compare instance.image_id to the id of image you expect.

To check if instance is already running, use EC2.Waiter.InstanceRunning waiter.

Answer 2

To check if an EC2 instance is up or down:

import sys
import boto3


ec2 = boto3.client('ec2')
if sys.argv[1] == 'ON':
    response = ec2.monitor_instances(InstanceIds=['INSTANCE_ID'])
else:
    response = ec2.unmonitor_instances(InstanceIds=['INSTANCE_ID'])
print(response)

To check the health of an instance:

# Boto 2.x
for status in ec2_connection.get_all_instance_statuses():
    print(status)

# Boto 3
for status in ec2.meta.client.describe_instance_status()['InstanceStatuses']:
    print(status)

For more details refer to: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/migrationec2.html

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/ec2-example-managing-instances.html

Update: (To check if particular service in EC2 is running or not)

1. You can publish a custom metric to CloudWatch in the form of a "heart beat".
2. Have a small script running via cron on your server checking the process list to see whether my_service is running and if it is, make a put-metric-data call to CloudWatch.
3. The metric could be as simple as pushing the number "1" to your custom metric in CloudWatch.
4. Set up a CloudWatch alarm that triggers if the average for the metric falls below 1
5. Make the period of the alarm be >= the period that the cron runs e.g. cron runs every 5 minutes, make the alarm trigger if it sees the average is below 1 for two 5 minute periods.
6. Make sure you also handle the situation in which the metric is not published (e. g. cron fails to run or whole machine dies). you would want to setup an alert in case the metric is missing. (see here:AWS Cloudwatch heartbeat alarm)
7. Be aware that the custom metric will add an additional cost of 50c to your AWS bill (not a big deal for one metric - but the equation changes drastically if you want to push hundred/thousands of metrics - i.e. good to know it's not free as one would expect)
8. You have to create a topic in SNS (Simple Notification Service) service in AWS. There you have to define your email under SNS on which you will receive the notification.

References:

1. Use cloudwatch to determine if linux service is running
2. AWS Cloudwatch Heartbeat Alarm

Answer 3

While you ask specifically about boto3, there is a way to ensure some service is in ready state more AWS natively.

Move creation of your instance to CloudFormation template and define CreationPolicy, like described in AWS Blog post: https://aws.amazon.com/blogs/devops/use-a-creationpolicy-to-wait-for-on-instance-configurations/

TL;DR: when the instance is launched, it will run a script which will check if whatever service you need is working, and then signal CloudFormation "all systems go". If it does not signal in time, instance is marked as CREATE_FAILED and deleted.