2

I have a Docker image containing Python code and third-party binary executables. There are only outbound network requests. The image must run hourly and each execution lasts ~3 minutes.

I can:

  1. Use an EC2 instance and schedule hourly execution via cron
  2. Create a CloudWatch Event/Rule to run an ECS Task Defintion hourly
  3. Setup an Elastic Beanstalk environment and schedule hourly deployment of the image

In all of these scenarios, an EC2 instance is running 24/7 and I am being charged for extended periods of no usage.

How do I accomplish scheduling the starting of an existing EC2 instance hourly and the stopping of said instance after the completion of my docker image?

Sean Pianka
  • 2,157
  • 2
  • 27
  • 43

1 Answers1

1

Here's one approach I can think of. It's very high-level, and omits some details, but conceptually it would work just fine. You'll also need to consider the Identity & Access Management (IAM) Roles used:

  • CloudWatch Event Rule to trigger the Step Function
  • AWS Step Function to trigger the Lambda function
  • AWS Lambda function to start up EC2 instances

  • EC2 instance polling the Step Functions service for Activity Tasks

    1. Create a CloudWatch Event Rule to schedule a periodic task, using a cron expression
    2. The Target of the CloudWatch Event Rule is an AWS Step Function
    3. The AWS Step Function State Machine starts by triggering an AWS Lambda function, which starts the EC2 instance
    4. The next step in the Step Functions State Machine invokes an Activity Task, representing the Docker container that needs to execute
    5. The EC2 instance has a script running on it, which polls the Activity Task for work
    6. The EC2 instance executes the Docker container, waits for it to finish, and sends a completion message to the Step Functions Activity Task
    7. The script running on the EC2 instance shuts itself down
    8. The AWS Step Function ends

Keep in mind that a potentially better option would be to spin up a new EC2 instance every hour, instead of simply starting and stopping the same instance. Although you might get better startup performance by starting an existing instance vs. launching a new instance, you'll also have to spend time to maintain the EC2 instance like a pet: fix issues if they crop up, or patch the operating system periodically. In today's world, it's a commonly accepted practice that infrastructure should be disposable. After all, you've already packaged up your application into a Docker container, so you most likely don't have overly specific expectations around which host that container is actually being executed on.

Another option would be to use AWS Fargate, which is designed to run Docker containers, without worrying about spinning up and managing container infrastructure.

  • I appreciate the high level overview of this method, it's given me lots of leads for learning and research as I'm a beginner to AWS. As far as the process for shutdown of the EC2 instance, couldn't I have a state which invokes a lambda function which terminates the EC2 instance I spawned in the first state? It'd be the "Next" after the state which invokes the Docker container. – Sean Pianka Jul 22 '18 at 16:53
  • 1
    Yes of course that's another option. I was just trying to keep the solution as simple as possible. –  Jul 22 '18 at 17:25
  • Would you recommend creating a new cluster each I want to run the docker image? I'm having difficulty spinning up a new instance (with an ecs-optimized image) and attaching it to an existing cluster. However, according to https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_AWSCLI_EC2.html, programmatically spinning up a cluster + instance is easy. – Sean Pianka Jul 22 '18 at 20:15
  • 1
    With either approach, you're still creating EC2 instances and joining them to a cluster. I'd try to hone in on the specific issue you're having with spinning up an instance and joining it to a cluster. Assuming your IAM instance profile / role and EC2 User Data is set up correctly, it should "just work." –  Jul 23 '18 at 13:43
  • I've gotten everything to work, including the launching of a new EC2 instance for each CloudWatch event triggered. I was able to condense each of the steps with launching the instance, attaching to the cluster, creating the CloudWatch logs for the task definition, running the task, and terminating the instance only after the task has completed into a single Lambda function (meaning, I was able to avoid using step functions). My method for polling is crude, using `time.sleep(...)` and querying via boto3 continually, but it does the job. Thank you for all your help! – Sean Pianka Jul 28 '18 at 15:28
  • One of the downsides to my method of using the lambda function for everything, including polling and waiting for the end of the task (rather than the docker container somehow triggering something to notify of its completed execution) is that it racks up usage time on the lambda function (since there's several minutes that it's simply polling to see if the task has completed). I wonder if step functions would offer an advantage there. – Sean Pianka Jul 29 '18 at 15:12
  • 1
    @SeanPianka Yes, you can certainly use AWS Step Functions for this purpose. In fact, I published a blog post that demonstrates this in action, where polling more often than 60 seconds is necessary. Since CloudWatch Events is limited to maximum precision of 60 second intervals, Step Functions can fill the gap here, and reduce the amount of waste compute time in AWS Lambda. See here: https://aws.amazon.com/blogs/compute/capturing-custom-high-resolution-metrics-from-containers-using-aws-step-functions-and-aws-lambda/ –  Jul 29 '18 at 20:39