0

I created a learning project using Spring Data Redis, bitnami/redis and bitnami/redis-sentinel images. The Docker Compose file is here. The deployment command is docker-compose -f redis-sentinel.yaml up --scale redis-sentinel=3 --scale redis-replica=2 -d, so the Redis configuration is as follows:

enter image description here

After bringing everything up, I stop the master container with docker stop redis-master. After 10 seconds, the sentinels failover to one of the replicas. So far so good. However, nothing is shown in the container console logs for the following very important things:

  1. master down has been detected.
  2. Failover has been initiated.
  3. Failover has successfully completed.

I was expecting the sentinel to log the aspects indicated above.

Also, when I run sentinel masters command from the Redis CLI of one of the sentinels, I see no difference in the number of replica before and after failover. I was expecting to see the number drop by 1 when the master is shut down.

31) "num-slaves"
32) "2"
33) "num-other-sentinels"
34) "2"
Abhijit Sarkar
  • 21,927
  • 20
  • 110
  • 219
  • Which log files did you check? Please ensure 1. Sentinel Logs are enabled 2. Make sure you are looking at sentinel logs and not master/slave logs. – Vikram Rawat Nov 16 '20 at 11:34
  • @VikramRawat I checked the sentinel Docker container logs. 1. How'd I know sentinel logs are enabled or not? 2. If I'm looking at the sentinel Docker container logs, I'd assume I'm looking at the right place. – Abhijit Sarkar Nov 16 '20 at 11:41
  • 1
    find the sentinel.conf file, which has the log file path defined in it .. the default value is logfile "", which logs to standard output – Vikram Rawat Nov 16 '20 at 11:45
  • @VikramRawat Found under `/bitnami/redis-sentinel/conf/sentinel.conf` and `logfile "/opt/bitnami/redis-sentinel/logs/redis-sentinel.log"`. `redis-sentinel.log` does show the failover. If you post your comment as an answer, I'll be happy to accept it. Would you also answer the 2nd part, which is when master is down, why replica count doesn't decrease? – Abhijit Sarkar Nov 16 '20 at 12:50
  • I have added the answer. Not sure on the second part though.. Unless you restarted the old master (new slave), it should have shown 1 slave. – Vikram Rawat Nov 16 '20 at 13:26

1 Answers1

1

Please ensure 1. Sentinel Logs are enabled 2. Make sure you are looking at sentinel logs and not master/slave logs. You can find the sentinel.conf file, which has the log file path defined in it .. the default value is logfile "", which logs to standard output.

Vikram Rawat
  • 1,472
  • 11
  • 16
  • For the record, I filed a bug https://github.com/bitnami/bitnami-docker-redis-sentinel/issues/34. The failover logs should go to stdout, and visible via `docker logs` command. – Abhijit Sarkar Nov 16 '20 at 13:53