2

I have been learning about the Docker and its advantages. These advantages include, but are not limited to:

  • Rapid Deployment
  • Portability
  • Security
  • Isolation
  • Version control
  • Lightweight footprint with minimal overhead

My questions:

  • Will it always be less stressful on a host machine to run multiple applications such as Zeppelin, Hadoop, Flume, ect. in individual docker containers or would the application virtual machine be added on top of the container (docker creates overhead)?

  • At some point does the number of containers running create an overhead that will cost more resources than it would to just run all of the tools directly off the host machine?

  • Would it be better to run all of the apps in one container?

Video about docker: https://www.youtube.com/watch?v=YFl2mCHdv24

ob1
  • 1,425
  • 1
  • 20
  • 33

1 Answers1

0

I found this forum reply by someone at Docker.

Will it always be less stressful on a host machine to run multiple applications such as Zeppelin, Hadoop, Flume, ect. in individual docker containers or would the application virtual machine be added on top of the container (docker creates overhead)?

The post seems to indicate Docker does come with overhead that scales linearly with the number of containers being ran on a given host system. This has to do with Docker being written in Golang and how Golang operates.

At some point does the number of containers running create an overhead that will cost more resources than it would to just run all of the tools directly off the host machine?

Judging from the aforementioned Docker staff response, it seems like this would be the case. I'd not look at "overhead" in isolation though; running 40 different Java containers with disparate versions on one host is unrealistic for technical reasons. Docker allows you to do this easily because it isolates each process.

So if I may do a slanted comparison, the human management overhead created by managing 40 Java applications on one host would certainly be greater cost than the additional system resources you might save by foregoing containerizing them.

Would it be better to run all of the apps in one container?

Assuming the context here is system overhead, and taking into consideration the linear scaling of Docker process overhead with running container count, fewer containers = less Docker process overhead. Similar to the previous question though, you might be introducing complexity by lumping everything into one container that would cost some poor soul a lot of hours troubleshooting or fixing.

If you have several pieces (web server, app server, database, memcache, activemq), putting them in one container will eventually become very inefficient because you can't scale them individually. If you need to scale your app server, you can't just scale your app server, you have to also unnecessarily scale all the other services in the container.

ob1
  • 1,425
  • 1
  • 20
  • 33
bluescores
  • 4,437
  • 1
  • 20
  • 34
  • Thanks for the thorough answer and for citing the reference! Would you have any idea of how to determine what number of containers (depending on the app running) it would create to reach that limit for a machine. I realize this number will vary on the different specs of ever machine. Im particularly interested in the ability of a raspberry pi 3! If you don't know off the top of your head would you know a good place for me to look for this answer? – ob1 Jul 11 '17 at 16:12
  • 1
    @ob1 You could try a simple docker-compose file, then using `docker-compose scale` to add containers one by one, monitoring the system resources on the Raspberry Pi as you do so. – bluescores Jul 11 '17 at 16:24