0

What's the best, simple way of deploying a lot of containers to AWS? It's necessary to be scalable but cheap if a container sees little or no use.

I have a method of generating docker containers that serve data (using fastAPI). Each container runs some basic numpy/ML, nothing too intensive. More importantly, they might not receive any requests for days on end, we need to able to deploy lots of these for minimal cost and not worry about keeping a whole EC2 instance up for them.

Appreciate anyone with an opinion on docker & AWS. Here's my understanding of the solutions (and why I'm not satisfied with them):

  • Elastic Beanstalk

This means uploading your container to ECR, then setting up a load-balancer environment on EB with an endpoint. This is the current solution I use, it's inconvenient that it's endpoint is a public URL, but I've already gone through the challenge of getting that on my virtual cloud. However, I suspect that while this can scale up as it needs to, it always keeps at least one EC2 instance running per container, which means paying like 1000x the cost of what you actually need, maintaining an EC2 instance that just sits there.

  • EC2 instances

I can just deploy an EC2 instance and run my containers on there. But this seeems extremely manual in terms of load (how many containers can fit on one instance?) and in terms of endpoints (I'd have to track the name and assign each container its own port), which seems messy and error prone.

  • Lambda Function uploaded as docker container

This means altering the docker container to implement a lambda_function method as it's api, then deploying it as a lambda function. This seems to be the only way to maintain containers in a zero-use means zero-cost (or minimal cost) fashion, paying only as requests come in. But, it means modifying the container for this use case and setting up a bunch of extra plumbing; I've got to somehow meld the typical lambda function dockerfile with my current web-api based dockerfile, and work out how to translate API Gateway requests into docker-lambda invocations.

  • Lambda Function, no longer using docker at all

Now I've completely deconstructed the docker containers, but maybe docker is just the wrong thing to use if you need 24/7 access but don't want to keep them running?

Conclusion

So basically I suspect I need to morph my containers into lambda-functions, which is doable but kind of defeats the purpose of having our contractors use docker (I'm now modifying the build procedure in a way that fits with my target environment, the exact thing docker is supposed to prevent!)

Paranoid Altoid
  • 21
  • 1
  • 3
    As you've pretty much worked out yourself, lambda functions may be the best way to do this. Any hosting using ECS Fargate / EC2 will cost money when it's idle. – Tim Jan 17 '23 at 00:14
  • Makes sense. I'll think about whether I can use the fastAPI docker build with lambda somehow, or if I should forget it. Speculating, containers take up a lot of space take a while to startup, so a lambda function that, say, uses docker run on each invocation might be inefficient. Maybe it's good enough having a python function that does "docker run" equivalent on import and uses the docker api in lambda_function would be okay if I understand lambda, doing docker run once if 100 invocations happen in a minute, and sleeping if not invoked for days. – Paranoid Altoid Jan 17 '23 at 00:35
  • Can you package the code directly for lambda, rather than using containers? Lambda SnapStart might help startup speed, though I'm not sure if it covers docker to start with as it's fairly new https://aws.amazon.com/blogs/aws/new-accelerate-your-lambda-functions-with-lambda-snapstart/ – Tim Jan 17 '23 at 02:47

0 Answers0