3

I was wondering if it's possible to run multiple docker registry hosts under an ELB or (any load balancer for that matter). I am planning to run the private docker registry inside a docker container. Has anyone tried that? Any ideas for how to have Docker registry containers autoheal without using cluster management tools like mesos, is that possible? For now with one docker registry container on an ec2 instance I could easily use an ELB for the health check and just tear down and spin up a baked AMI with the docker registry. But just curious if I can run a clustered setup for a docker registry (multiple docker regsitry containers on multiple ec2s or a single high end ec2.

alexfvolk
  • 1,810
  • 4
  • 20
  • 40

2 Answers2

3

We're currently performing testing of a private Docker registry behind an AWS ELB, and it appears the ELB is modifying the request/response in a way that prevents this from working properly (tcp connection timeouts are the symptom). We're still performing testing, but we may simply use DNS load balancing with service checks if an ELB is unable to properly proxy connections to the registry.

EDIT: As of Monday, March 23rd 2015 it appears that AWS ELBs can properly pass docker traffic between clients and backend Docker registry instances.

EDIT 2: We've moved away from managing our own Docker registry in AWS and now use the AWS container registry. We've found this to be a superior experience if you're in need of a container registry while running within AWS.

Brandon Galbraith
  • 474
  • 5
  • 8
  • @alexfvolk It appears that AWS made a change to their ELB setup. I tested again this evening, and an ELB is successfully passing docker client push/pull container requests and payload data through to my backend docker registry instances. – Brandon Galbraith Mar 24 '15 at 02:42
  • Thanks Brandon, Another question - Do you use SSL/authentication using nginx for your docker registries or do you have a different setup with ELB providing the SSL piece? – alexfvolk Mar 24 '15 at 17:09
  • I'm using docker registry authentication, which relies on the auth contained in `~/.dockercfg` on the client-side, with the ELB providing the SSL termination. – Brandon Galbraith Mar 25 '15 at 17:18
  • What kind of listener did you use for the ELB ? HTTP or TCP directly ? – golja Mar 26 '15 at 14:54
  • @golja HTTP listener (HTTPS served, connecting to HTTP on the instance) – Brandon Galbraith Mar 26 '15 at 20:28
  • Aha I see ... I noticed that if you use a tcp listener with the backend, ELB will always keep the session with the same backend as long as the connection is kept alive. Probably this could also solve the issue alexfvolk was experiencing. – golja Mar 27 '15 at 12:50
  • I am a little confused about the docker registry authentication on the host side. I thought you have to setup nginx for that and use apache2utils to create the htpasswd file etc. What does your setup look like? – alexfvolk Mar 27 '15 at 22:27
  • Please help me solve this http://stackoverflow.com/questions/36295707/blob-upload-unknown-error-in-clustered-docker-private-registry – Casper Mar 29 '16 at 21:50
0

Maybe not exactly your use-case BUT: We wanted to improve the availability of the Docker registry as well. Our main concern was reading, since auto-scaling spins up new instances very often. It turned out that the most stable solution in terms of maintainability and stability was to use s3 as the backend and start a registry docker image on every instance. You basically pull from localhost and have the full stability of s3 as backend. Without the need for load-balancing a central registry (point-of-failure).

Maybe not suitable for everybody but worked well for us.

MohammadHosseinZiyaaddini
  • 658
  • 1
  • 6
  • 17
Jan
  • 2,803
  • 6
  • 36
  • 57