2

I have three replicated containers running RabbitMQ in a Swarm Mode service. Each container acts as a RabbitMQ node to create a RabbitMQ cluster. The docker service create for this is as follows:

docker service create -e RABBITMQ_ERLANG_COOKIE='mysecretcookie' --replicas 4 --network rnet -p 15672:15672 -p 5672:5672 rabbitmq

I'm able to successfully create a RabbitMQ cluster using rabbitmqctl <hostname> join_cluster on each of the slave nodes thereafter.

My problem is that if a swarm node goes down, Docker starts a new container on a different node however the container has a different random hostname.

Since RabbitMQ uses the hostname to identify RabbitMQ cluster nodes, it doesn't recognise the new hostname on the new container so it assumes that the original node is indefinitely down.

I tried using the new templating feature in Docker 1.13 which allows you to create static hostnames by specifying --hostname="{{.Node.ID}}-{{.Service.Name}}" within docker service create. However, you cannot currently discover containers based on this custom hostname so a RabbitMQ cluster can't be created in this way.

I want to be able to have RabbitMQ automatically re-join a cluster node after Docker starts running a new container. Is this possible?

AmazingBergkamp
  • 5,708
  • 4
  • 15
  • 24

1 Answers1

0

I want to be able to have RabbitMQ automatically re-join a cluster node after Docker starts running a new container. Is this possible?

I assume you mean "re-join a cluster node existing in the newly started docker container".

The answer - it's not. Because, by definition, it would not be re-joining but simply joining since, from RabbitMQ's perspective, this is simply a new machine (computer, vm, docker...) in the network.

Pang
  • 9,564
  • 146
  • 81
  • 122
cantSleepNow
  • 9,691
  • 5
  • 31
  • 42
  • Yes, that's what I meant. Apologies, it's not clear. Basically, I want to synchronise the Docker swarm replicas and the RabbitMQ cluster nodes between host/swarm node failures - since any failed RabbitMQ nodes are still listed by RabbitMQ just as 'Node not running' so I assumed that there was some way of bringing these back up in conjunction with Docker's ability to start new containers if it spots a node failure – AmazingBergkamp Jan 25 '17 at 13:36
  • Maybe there is some docker container management software that does this,or you can write a script - so for restarting. For automatically starting you could create your own image that will join to predefined cluster. – cantSleepNow Jan 25 '17 at 21:18