0

So basically I have five tasks that need to be run once a day and the process is usually quick.

The problem I am having is that I don't want to have an EC2 up all day just to use it for thirty minutes. Is there a way to have the ECS create an instance, run the tasks and finish the instance until the next day?

Mahdi
  • 3,199
  • 2
  • 25
  • 35
Marco
  • 1,112
  • 1
  • 14
  • 34
  • Is this what you are looking for: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduled_tasks.html ? – Mahdi Jul 20 '18 at 11:10
  • @Mahdi No, im looking to automatically start and stop the EC2 instance only for when the tasks need to run and thats it. Thanks anyway. – Marco Jul 20 '18 at 12:30
  • Have you looked at the AWS CLI? You can easily create scripts to start and stop EC2 instances. Example to start an instance: aws ec2 start-instances --instance-ids i-1234567890123456. You could then script SSH commands to run in your instance and then stop it. All from the command line (or via a scheduler). – John Hanley Jul 21 '18 at 07:24
  • One item to remember. EC2 instance cost per hour and disk storage cost (EBS or instance storage). For small EC2 instances the storage cost can be larger than the instance cost. You cannot turn of the storage cost. – John Hanley Jul 21 '18 at 07:26

1 Answers1

2

These are a few options:

1. Scheduled Scaling with an Autoscaling Group

You run your instance in an autoscaling group and use time based scaling to set the desired count to 0 when you don't need it and 1(or any number you want) when you have tasks to run.

https://docs.aws.amazon.com/autoscaling/ec2/userguide/schedule_time.html

2. Use AWS Instance Scheduler

AWS Instance Scheduler allows you to create EC2 start-stop scheduling rules. In this approach, you configure the schedule in a dynamodb table, add an appropriate tag for your instance and then a lambda function will ensure that the schedule is followed.

https://aws.amazon.com/answers/infrastructure-management/instance-scheduler/

3. EC2 Start Stop based on pending task count

Schedule a lambda function to run periodically to detect the pending task count for your cluster. If it is non zero, start the instance. The lambda should also stop the instance when there are no pending or running tasks. You'll need to use DescribeTasks API to filter tasks with lastStatus=PENDING

Some points:

  • If your tasks don't take a long time to complete, Fargate(if available) could be a better and cheap solution to run your tasks without worrying about servers.

  • You could also possibly use Cloudwatch alarm actions or AWS Batch but one of the above options is probably simpler and enough for your use case.

user818510
  • 3,414
  • 26
  • 17