4

I use the K8s example the create a Reliable, Scalable Redis on Kubernetes. https://github.com/kubernetes/kubernetes/tree/master/examples/redis

The Redis master needs to get slave the origin IP and register to the sentinel. Sentinel will use to switch master when the master failed.

But My docker starts without the flag iptables=false(It will create an iptables like SNAT), so the redis slave source IP will be SNAT, and the master will get the flannel0's IP like 172.16.103.0.

I also add the flannel network to container connection between nodes.

So the question is "Is any way to get the source IP without removing the iptables flag, or another way to create a Reliable redis cluster."

AATHITH RAJENDRAN
  • 4,689
  • 8
  • 34
  • 58
zhulinhong
  • 51
  • 1
  • 3

3 Answers3

2

EDIT: I have figured out the magic combination of flags to get this working.

1) run stop docker & the bootstrap docker processes 

systemctl stop docker && pkill -f "docker-bootstrap" && sleep 10

2) Delete iptables rules: 

iptables -F && iptables -t nat -F

3) run your docker-daemon with the ip-masq=false option
4) run your flanneld with the ip-masq=true option

For me this makes all three pod->pod, pod->ext, & ext->service->pod work perfectly.

Hope this works for the rest of you!

(Preserving initial response which described the symptoms)

I am having the same problem with the guide. The slave replicates just fine but the master can't properly check the slave status because the connection is coming from the dot zero ip. 

Proto Recv-Q Send-Q Local Address           Foreign Address         State

Example slave->master connection from the master perspective (shows wrong IP of slave):

tcp   0      0 redis-master:6379       10.1.37.0:51674         ESTABLISHED

Example slave->master connection from slave perspective (shows right ip of master):

tcp   0      0 redis-03fdy:51674       10.1.90.2:6379          ESTABLISHED

Here are the redis server commands from ps.

master:

root   12  0.1  0.2  26876  2352 ?   Sl   10:42   0:02 redis-server 10.1.90.2:6379

slave:

root   18  0.1  0.2  26876  2300 ?   Sl   10:48   0:00 redis-server 10.1.37.3:6379
ssjcory
  • 171
  • 1
  • 4
  • we're seeing the same thing, did you make any progress? – Andy Smith May 27 '16 at 09:56
  • @AndySmith yes, please see my adjusted answer :) P.S. make sure flannel ip-masq is running on all nodes – ssjcory May 31 '16 at 12:59
  • I try this way, but "ping" failed, in container. – zhulinhong Jun 02 '16 at 10:27
  • @zhulinhong Are you sure flannel is running and using the same configurations (for example I have --etcd-endpoints=http://${MASTER_IP}:4001) ? Also when you start docker you started with the flannel subnet & mtu ? You can use docker logs $container_id on the containers running on any node get more info about what could be going wrong. I suggest doing that on the flannel container. – ssjcory Jun 03 '16 at 12:57
0

We managed to work around this by setting hostNetworking: true in our redis pods. This means the redis servers don't get container IPs (172..) and is definitely a work around but fixes the issue.

Andy Smith
  • 3,308
  • 4
  • 26
  • 34
0

Here is how you can setup Redis HA Master Slave Cluster in Kubernetes/Openshift OKD, without using Helm, and sentinal not required

Basically you have to use configMap, StatefulSet in collaboration with VolumeClaims

craftsmannadeem
  • 2,665
  • 26
  • 22