1

I'd like to set up a production HA Docker Swarm cluster.

Docker Swarm needs a Key Value store as a precondition, consul in this case.

Now I'm asking what a clever setup would be:

  • Consul should run in Docker containers
  • Consul must be HA and automatically scaleable
  • Consul containers should be managed with Docker Swarm (this would need consul running though)

Constraints

  • Running consul on the host is not an option
  • Having to manage consul containers manually is not an option
  • Docker Swarm with a Docker Hub token is not an option

Approach

Any ideas how this chicken or the egg problem can be solved in a clean and reliable manner for efficient operation?

Multiple swarm instances would certainly look like a reasonable approach but how to setup the initial swarm without consul and swarm? Thanks.

030
  • 5,901
  • 13
  • 68
  • 110
binaryanomaly
  • 406
  • 1
  • 4
  • 14
  • 2
    Docker Swarm is "still in its infancy" as what little documentation it has says. And large parts of that documentation appear to not yet be written, including what you are looking for. I don't think this is production ready. – Michael Hampton Jan 10 '16 at 20:40
  • Thanks for your comment. Docker claims it is [production ready](http://blog.docker.com/2015/11/swarm-1-0/). I'm not that naive to not understand that this is more of a _marketing production ready_ and you certainly have a point being skeptic and pointing out the lack of documentation as a proof. Nevertheless I hoped that somebody already encountered and solved this very fundamental question in a proper way. I hope there's already more existing than just the hype... – binaryanomaly Jan 10 '16 at 20:55
  • There's always been a lot more hype than reality surrounding Docker. I didn't even seriously consider Swarm for a production anything; at the moment Kubernetes and OpenShift v3 (which is based on Kubernetes!) are the only things that I am aware of that are ready to go. – Michael Hampton Jan 10 '16 at 21:06
  • Regarding the hype that is certainly the case. But with Kubernetes still being only single master I would also not consider it being fully _battle tested_ and rely on it. Mesos might be but it feels too heavy and not optimal for lighter setups. But even for setting up Kubernetes I would prefer to deploy it in containers and for that basic setup swarm could be a nice minimalistic solution since it doesn't need much more than a key value store. Since already the very basic set up of swarm in containers currently appears as not properly solved it may take some more time though. – binaryanomaly Jan 10 '16 at 21:33
  • Multiple masters is one reason I went with Openshift (not to mention multiple etcds). – Michael Hampton Jan 10 '16 at 21:39
  • Actually I think that [RancherOS](http://rancher.com) has atm best solved this challenge by providing a system-docker and a user docker instance, interesting concept. – binaryanomaly Jan 20 '16 at 18:35

1 Answers1

2

Jacob Blain Christen has a nice article about this very problem. In short: start the docker daemon with a reference to the not yet existing consul cluster, then start consul via docker. Citation:

Fortunately, the Docker daemon will happily retry to connect to the cluster-store aka the KVS every so often, this gives us time to underlay it via Docker Compose.

Hope this helps.

Swen Thümmler
  • 36
  • 2