0

I have deployed a NER model using the docker container on EC2. The generated docker image occupied 3GB with all the dependencies. I have a few questions regarding deploying multiple ML models.

  1. Do I need to create an image for each application and orchestrated it with Kubernetes?
  2. If so, is there a way to have the global environment dependencies for all the applications. I think creating 2GB of docker images for each application is an inefficient approach.
  3. What are the common practices followed in this kind of scenario?
Knilakshan20
  • 59
  • 6
  • How much of those "dependencies" are the model data? Can you distribute the model data separately from the rest of the application, and inject it using a Docker bind mount? (This is a little harder to manage if you do use Kubernetes – not at all required – but using an AWS EFS volume could help there.) – David Maze Feb 19 '22 at 18:33
  • With PyTorch GPU version and its associates, it took 2GB. How to distribute the model data separately? – Knilakshan20 Feb 21 '22 at 04:23

0 Answers0