10

When hosting applications on Heroku I often trigger one-off dynos via the Heroku API from within the code to do heavy lifting in the background. I recently set up some stacks on AWS and followed a similar pattern by using AWS ECS run task.

I am not using long running queue workers for this as hardware resources vary heavily according to the specific task and usually the workload occurs in peaks.

For local development, I usually skipped this topic by either executing the background tasks within the running container or triggering the background command manually from the console. What would be a good approach for running one-off containers locally?

André
  • 2,042
  • 1
  • 23
  • 26
  • How about triggering 'heroku local:run' from within your code? – Yoni Rabinovitch May 23 '18 at 10:06
  • I am running PHP. How could I run that from the container without being a child process of the container that triggers the process? Besides that I was also looking for a generic solution to this issue. – André May 23 '18 at 19:16
  • Perhaps you can run a second container in your local dev env, in which you would have a 'spawner' process responsible for spinning up 'heroku local:run' processes, based on input received from your PHP in your main container? So, on Heroku you use the platform API to spin up a one-off dyno, whereas on dev you trigger an auxiliary container to spin up a 'heroku local:run' process. – Yoni Rabinovitch May 24 '18 at 06:29
  • Sounds like you would need to have your first container hit the AWS api and create the second container on ECS – kevzettler Jun 01 '18 at 22:28

3 Answers3

0

ECS supports scheduled tasks, if you know when your peaks are planned for you can use scheduled tasks to launch fargate containers on a schedule.

If you don't, what we did was write a small API Gateway -> Lambda function that basically dynamically launches fargate containers with a few variables defined in the POST to the API Gateway endpoint like CPU/Mem/port etc...Or pre-create task definitions and just pass the task def to the api, which is another option if you know what the majority of your "settings" should be for the container.

stobiewankenobi
  • 694
  • 1
  • 10
  • 16
0

You can simply call ECS RunTask API call from inside the container.

All you need is to setup ECS Task role to have runtask permissions and to have either aws cli or any aws sdk in container to call runtask call.

Mangal
  • 607
  • 4
  • 8
0

you can pass docker socket as a volume

volumes:
     - /var/run/docker.sock:/var/run/docker.sock

After it you can run docker commands inside the container and they will be executed by docker on the host machine.

In particular you can run

docker run ...

or

docker start ...

(may be you will have to install docker in your container via commands in Dockerfile)

Ryabchenko Alexander
  • 10,057
  • 7
  • 56
  • 88