0

I have a docker swarm consisting of one manager and one worker.

When I created a service with 4 replicas, I got two containers on each node (2 on the manager and 2 on the worker node).

  • I restarted the worker node, so I found the manager now has all the load (the 4 containers).
  • After restart, the manger still has 4 containers and the worker has nothing.

  • Shouldn't the load be re-distributed automatically after the worker node is back from restart?

  • If not, how can I re-distribute the load?

Ehab
  • 566
  • 6
  • 24

1 Answers1

2

With only a single manager, you have no HA and when it's down, nothing will be rescheduled. You need at least 3 managers to support a single down manager scenario. When the single manager restarts, it's a race condition between the scheduler assigning workload, and the existing node reconnecting, that a worker node is unlikely to win.

Swarm mode will not proactively rescale reschedule workloads either, when adding new nodes to the cluster, or just restarting a down node, existing tasks will continue to run in their current node until there's a change that forces them to be rescheduled. That change can be a down node, or an update to a service. This improves HA since a new node may be unstable/flapping.

To force swarm mode to rebalance a service, you can run:

docker service update --force $service_name

This will force an update without any other modifications to that service. (Replace $service_name with the name or id of your service.)

BMitch
  • 231,797
  • 42
  • 475
  • 450