3

When I'm running docker registry in swarm:

docker service create \
  --name docker-registry \
  --mount type=bind,src=/some/path,dst=/var/lib/registry \
  -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
  --publish 5000:5000 \
  --replicas 2 \
  registry:2

and trying to push or pull to that registry (from another server) it hangs:

$ docker tag hello-world:latest 111.222.333.333:5000/hello-world
$ docker push 111.222.333.333:5000/hello-world
The push refers to a repository [111.222.333.333:5000/hello-world]
428c97da766c: Retrying in 1 second

but it works when I run it as container:

docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name docker-registry \
  -v /some/path:/var/lib/registry \
  registry:2

I added IP registry to insecure registries. What am I doing wrong?

Taz
  • 5,755
  • 6
  • 26
  • 63
  • You are pulling from 0.0.0.0:5000 however, you are trying to push to 111.222.333.333:5000? Did you made sure that you are pushing to the right registry also when you run docker login what happens? – Sergiu Oct 27 '18 at 10:58
  • According to manual https://docs.docker.com/registry/deploying/#run-the-registry-as-a-service `0.0.0.0:5000` is just to set port of registry. I'm pushing/pulling from another server to/from correct registry - I'm sure about that. And I do not use login, it's open registry within my private network. – Taz Oct 27 '18 at 11:06
  • 2
    @Taz, you have set `--replicas 2` for service, and they mount `/some/path` from host. Are you sure that both replicas run on same physical host? If they are on different, then your requests will be loadbalanced between them, generating inconsistent data. For instance, `push` will be routed to replica1 on host1, but `pull` will be routed to replica2 on host2. I would suggest running `registry` service with at least `--replicas 1`. Then in any case, if that replica restarts on the host2 the other day, it will start with fresh empty registry. – muradm Oct 27 '18 at 11:12
  • 1
    Either choose shared storage if you have, or make sure that your `registry` service always runs on same host, mounting same `/some/path`. As an option, if you are in Swarm mode, you could make your network `--attachable` and create `registry` not as service, but as conventional container. – muradm Oct 27 '18 at 11:15
  • Changing to `--replicas 1` solved the problem :) So I'm not sure now how scaling works, because both of them are on same host so I thought that when 2 pulls or pushes will occur concurrently every process will get one replica to not to block second one. – Taz Oct 27 '18 at 11:22
  • @Taz, just make sure, that service always runs on the same host. Because, if you restart service, docker swarm may re-schedule it to different host, where `/some/path` will be empty, and you will have to re-publish your images. Service can be sticked to specific host via conditional labels as far as I remember. – muradm Oct 27 '18 at 13:10

1 Answers1

2

You have set --replicas 2 for service, and they mount /some/path from host. That means either:

  • 2 docker-registry containers on the same host accesses /some/path
  • 2 docker-registry containers on different hosts accesses /some/path

In both cases, how docker-registry manages the local storage, is not compatible with such configuration. Swarm will distribute all API requests among instances, causing data corruption and unexpected behavior.

One have to make sure that docker-registry running on the same host with same /some/path always.

For high availability configurations, there are mentions in documentation.

Kevin Kopf
  • 13,327
  • 14
  • 49
  • 66
muradm
  • 1,973
  • 19
  • 30