3

I have been searching alot on how to deploy redis with high availability on kubernetes. I have some problems using redis cluster mode and when using the master-slave mode we need to also deploy sentinel to be able to handle master failures

I have been reviewing this great doc which explains how to do that but, I think there is something missing.

I have deployed what's mentioned there but, I needed to make some changes to for the sentinel containers to run in sentinel mode now the main redis-master pod manifest which has the main redis master and the sentinel looks like this. 

# redis-master.yaml

apiVersion: v1
kind: Pod
metadata:
  labels:
    name: redis
    redis-sentinel: "true"
    role: master
  name: redis-master
spec:
  containers:
    - name: master
      image: redis
      env:
        - name: MASTER
          value: "true"
      ports:
        - containerPort: 6379
      resources:
        limits:
          cpu: "0.1"
      volumeMounts:
        - mountPath: /redis-master-data
          name: data
    - name: sentinel
      image: redis
      command:
        - redis-sentinel
        - "/redis-master-data/redis.conf"
      env:
        - name: SENTINEL
          value: "true"
      ports:
        - containerPort: 26379
      volumeMounts:
        - mountPath: /redis-master-data
          name: data
        - mountPath: /redis-master
          name: config
  initContainers:
  - name: copy
    image: redis
    command: ["bash", "-c", "cp /redis-master/redis.conf /redis-master-data/"]
    volumeMounts:
    - mountPath: /redis-master
      name: config
    - mountPath: /redis-master-data
      name: data
  volumes:
    - name: data
      emptyDir: {}
    - name: config
      configMap:
        name: example-redis-config
        items: 
        - key: redis-config
          path: redis.conf

now after all that I am having 2 problems

  • first this doc is making a service for the sentinels only so I know I should make a service for the redis but I am not even sure are they both masters or not.

  • second problem assuming one is master and the other is a slave when a failure happens and the sentinel elects a new master how to make that new master belong to the service of the redis masters and not the slave service (cause usually we will make 2 services one which expose masters and the other for slaves)

NOTE: please review the doc mentioned above to understand my question well.

E_net4
  • 27,810
  • 13
  • 101
  • 139
ElGenius
  • 135
  • 1
  • 1
  • 11
  • 1
    I would suggest you use redis operator https://github.com/spotahome/redis-operator. I've experienced a lot of failover issues with both sentinel mode and cluster mode. Redis uses ip address to communicate between sentinel and master/slave node. Once there is an ip change, the node will lost until you reset the sentinel cluster manually and the ip address of a pod is ephemeral . – Hang Du Sep 17 '19 at 03:31

2 Answers2

1

This may be very late but from my understanding you don't run a service for redis itself. You're running replication of redis sentinel services so your client can connect to any of them to query master redis pod's ip address.

each pod with master has a sentinel which talks to sentinel service(s). Learning updating about the status of their master. so whenever a failover happens, sentinel updates sentinel service, sentinels select a new master and update sentinels with this info again. so whenever client queries sentinel again, it will get new master's ip address.

shadowyman
  • 215
  • 3
  • 11
0

Best implementation for REDIS+K8S this far I have come across is - this BITNAMI REDIS HELM

E_net4
  • 27,810
  • 13
  • 101
  • 139
user13424620
  • 134
  • 5