Check the EC2 instance in auto scaling group before shut downing as whether any user working on it or not

Question

I need to do autoscale in AWS that, if cpu threshold reaches 80%(Eg..,) then automatically new instance should launch ..,( this is usual process) but in this case old instance(current instance , in which the cpu limits reaches the mentioned threshold ) get shut downed irrespective of the users (developer) working/logged on it; which leads to loss of data. what is the solution to avoid this.? is there any way by using AWS-Lambda to over come this issue along with auto scaling concepts or any other ways?

Answer 1

In the blog Using AWS Lambda with Auto Scaling Lifecycle Hooks you'll find an example on how to use AutoScaling Lifecycle Hooks to trigger Lambda via SNS and execute custom operations.

You could for example have a snippet in your login process (.bashrc, .profile...) that sends a message to SQS for a given instance, and removes it at logout, the triggered Lambda could then check if the instance is marked as "used" and wait for it to be freed.

Answer 2

With AWS auto scaling, here are five strategies to avoid loss of data:

1-Cloudwatch

You should become familiar with Cloudwatch. Cloudwatch will allow you to create alarms for metrics that define when an EC2 instance is removed from your Auto Scale Group. For example, you create a metric for CPU utilization and a metric for number of users. You set your auto scale policy to not terminate (aka Scale In) EC2 instances from your auto scale group unless the metrics fall below 50% CPU and has 0 users. Also, Cloudwatch can let you know if you have gone over your budget.

2-Scale In Lifecycle Hook

As you terminate and EC2 instance, you can use a Lifecyle hook to perform an action you define. Specifically the autoscaling:EC2_INSTANCE_TERMINATING lifecycle hook could be set to run a script to copy user data (or log files, etc) from the instance before it terminates.

3-Avoid Instance Stores

When you choose your AMI for your instances, use EBS backed AMIs. This allows your data to persist after the instance is terminated. Alternatively store your data in S3, or a AWS data store.

Avoid storing user data on an Instance Store (aka ephemeral storage), because that data is lost once the instance is done with termination during scale in or in the case of instance failure.

4-Manual Auto Scaling

Using the console or the CLI, manually scale your auto scale group. The can help avoid accidental data loss. One you remove the user data from the instances, you can manually scale down. Manual scaling can be a safe scaling fall back strategy as you decide/test a new design of your application.

5-Application Redesign

When using the cloud, it is best to treat your instances as replaceable at any moment. Design for failure (see lesson #2 at link). Strive to make your system as stateless as possible. Stateless design lends to less instance storage (typically) and lends more to horizontal scaling that you desire in your auto scaling group. (Search "Stateless Applications" in AWS Best Practice White Paper)