4

We run a container environment (Kubernetes) and we have a set of redis sentinels that watch over a bunch of redis instances.

Since it's a containerized environment, configuration is mostly dynamic. A sentinel container might die, another one replaces it, etc.

This poses a problem for application configuration. Normally on a static setup, you provide the client with all the addresses for the sentinels and he works with it. On a frozen container, if the environment change, the configuration becomes outdated.

To solve this, we can use a load balancer in front of the redis sentinels. This way even if the underlying containers/ips change, the application configuration is still valid.

I'm aware that sentinels never forget other sentinels (and the same for slaves) but we can flush those when changes do happen.

We do use this today, and haven't felt any side-effects AFAIK, but of course I'd like to know if there's a risk of something going wrong because of this.

So the question is: can I use a load balancer in front of redis sentinels without any major issues?

Draiken
  • 3,805
  • 2
  • 30
  • 48
  • that's what Services do: thye proxy the traffic to the Pod with the specified tag. You need to setup a Service for the Redis master(s) to access the store and a Service for sentinels to look up the masters (a headless Service will give you a list of masters.) Check out: https://github.com/kubernetes/kubernetes/tree/master/examples/storage/redis – MrE Dec 21 '16 at 00:18
  • @MrE I know that and as I said it's what we do today. My questions is whether or not using a load balancer in front of sentinels can cause issues. – Draiken Dec 21 '16 at 13:17
  • if i recall correctly, the script reconfigures the cluster and let the master and other sentinels know when a new pod is started. you could also use postStop amd preStart scripts, or now Init containers to do whatever you would otherwise do manually – MrE Dec 21 '16 at 16:51
  • I don't see any major problem, as kubernetes service support liveness/readiness, however, Redis client has to be able to handle command execution failure or disconnect, and do retry/re-connect accordingly, this beyonds the discussion here. I'm not running Redis in kubernetes, but we used to use haproxy as LB in non-kubernetes environment, it just works and I believe kubernetes service is about the same. – Hang Jan 05 '17 at 03:05

0 Answers0