we have a redis-sentinel cluster consisting of three VMs running redis and sentinel, and I am using redisson library to connect to it. However we are facing an issue when connecting to it. It seems that when it gets the list of sentinels, it gets the master as non ssl.
Versions: redis 6 java 1.8 redisson 3.16.4
here is the sentinel configuration on the VMs,
port 0
tls-port 26379
tls-cert-file "/etc/ssl/redis/redis.crt"
tls-key-file "/etc/ssl/redis/redis.key"
tls-ca-cert-file "/etc/ssl/redis/ca_uat.crt"
tls-replication yes
tls-auth-clients yes
daemonize no
pidfile "/var/run/redis-sentinel.pid"
sentinel myid e93ca95c2bd17723461f3d2a84296934f560c46e
sentinel deny-scripts-reconfig yes
sentinel monitor redis-cluster 182.26.3.54 6379 2
sentinel down-after-milliseconds redis-cluster 5000
logfile "/var/log/redis/sentinel.log"
dir "/tmp"
requirepass password
# Generated by CONFIG REWRITE
protected-mode no
supervised systemd
user default on #1d38e11ad3cff25192a679ae05dd0cebaa746da9185a66f719ff19311472a02f ~* +@all
sentinel failover-timeout redis-cluster 60000
sentinel auth-pass redis-cluster password
sentinel config-epoch redis-cluster 70
sentinel leader-epoch redis-cluster 70
sentinel known-replica redis-cluster 182.26.3.21 6379
sentinel known-replica redis-cluster 182.26.3.51 6379
sentinel known-sentinel redis-cluster 182.26.3.54 26379 2294c0239d9137af32e5d4b0fc2836e12727f13a
sentinel known-sentinel redis-cluster 182.26.3.51 26379 e69c7f50bb633a04981a20f54c7f5edcd46fc0c8
sentinel current-epoch 70
and here is the the non default redis config.
port 0
tls-port 6379
tls-cert-file /etc/ssl/redis/redis.crt
tls-key-file /etc/ssl/redis/redis.key
tls-ca-cert-file /etc/ssl/redis/ca_uat.crt
tls-replication yes
tls-auth-clients yes
masterauth password
requirepass password
here is the redisson yaml config
---
sentinelServersConfig:
idleConnectionTimeout: 100000
connectTimeout: 100000
timeout: 30000
retryAttempts: 3
retryInterval: 1500
password: "password"
subscriptionsPerConnection: 5
sslEnableEndpointIdentification: false
sslProvider: "JDK"
sslTruststore: "file:/resources/redis-cluster.jks"
sslTruststorePassword: "Jpass"
sslKeystore: "file:/resources/redis-cluster.jks"
sslKeystorePassword: "Jpass"
sslProtocols:
- "TLSv1.2"
pingConnectionInterval: 30000
keepAlive: false
tcpNoDelay: true
nameMapper: !<org.redisson.api.DefaultNameMapper> {}
loadBalancer: !<org.redisson.connection.balancer.RoundRobinLoadBalancer> {}
slaveConnectionMinimumIdleSize: 24
slaveConnectionPoolSize: 64
failedSlaveReconnectionInterval: 3000
failedSlaveCheckInterval: 180000
masterConnectionMinimumIdleSize: 24
masterConnectionPoolSize: 64
readMode: "SLAVE"
subscriptionMode: "SLAVE"
subscriptionConnectionMinimumIdleSize: 1
subscriptionConnectionPoolSize: 50
dnsMonitoringInterval: 5000
sentinelAddresses:
- "rediss://redis03.corp.com:26379"
- "rediss://redis02.corp.com:26379"
- "rediss://redis01.corp.com:26379"
natMapper: !<org.redisson.api.DefaultNatMapper> {}
masterName: "redis-cluster"
sentinelPassword: "password"
database: 0
scanInterval: 1000
checkSentinelsList: true
checkSlaveStatusWithSyncing: true
threads: 16
nettyThreads: 32
codec: !<org.redisson.codec.MarshallingCodec> {}
referenceEnabled: true
transportMode: "NIO"
lockWatchdogTimeout: 300000
reliableTopicWatchdogTimeout: 6000000
keepPubSubOrder: true
useScriptCache: false
minCleanUpDelay: 5
maxCleanUpDelay: 1800
cleanUpKeysAmount: 100
nettyHook: !<org.redisson.client.DefaultNettyHook> {}
useThreadClassLoader: true
addressResolverGroupFactory: !<org.redisson.connection.DnsAddressResolverGroupFactory> {}
and here is the code to create the connection
try {
reddisonConfig = Config.fromYAML(configFile);
logger.warn(reddisonConfig.toYAML());
} catch (IOException e) {
logger.warn("Unable to load configuration file");
logger.warn("file location is" + configFile.getAbsolutePath());
setConnected(false);
utilityClass.logMe(logger, e);
}
try {
Rclient = Redisson.create(reddisonConfig);
setConnected(true);
} catch (Exception e) {
logger.warn(
"Problem connecting to redis ");
setConnected(false);
utilityClass.logMe(logger, e);
}
and here is the error I get when I start the service.
2021-11-15 12:36:20.652 [restartedMain] INFO org.redisson.Version : Redisson 3.16.4
2021-11-15 12:36:21.008 [restartedMain] INFO o.r.c.SentinelConnectionManager : master: redis://182.26.3.21:6379 added
2021-11-15 12:36:21.016 [restartedMain] INFO o.r.c.SentinelConnectionManager : slave: rediss://182.26.3.54:6379 added
2021-11-15 12:36:21.016 [restartedMain] INFO o.r.c.SentinelConnectionManager : slave: rediss://182.26.3.51:6379 added
2021-11-15 12:36:21.094 [redisson-netty-11-10] INFO o.r.c.SentinelConnectionManager : sentinel: rediss://182.26.3.51:26379 added
2021-11-15 12:36:21.094 [redisson-netty-11-11] INFO o.r.c.SentinelConnectionManager : sentinel: rediss://182.26.3.54:26379 added
2021-11-15 12:36:21.102 [redisson-netty-11-9] INFO o.r.c.SentinelConnectionManager : sentinel: rediss://182.26.3.21:26379 added
2021-11-15 12:36:51.244 [restartedMain] WARN c.b.p.utilities.CacheUtil : Problem connecting to redis server
2021-11-15 12:36:51.245 [restartedMain] ERROR c.b.p.utilities.CacheUtil : Exception occurred at restartedMain Exception Message
org.redisson.client.RedisConnectionException: Unable to connect to Redis server: 182.26.3.21/182.26.3.21:6379
at org.redisson.connection.pool.ConnectionPool$1.lambda$run$0(ConnectionPool.java:158)
at org.redisson.misc.RedissonPromise.lambda$onComplete$0(RedissonPromise.java:183)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
at org.redisson.misc.RedissonPromise.tryFailure(RedissonPromise.java:96)
at org.redisson.connection.pool.ConnectionPool.promiseFailure(ConnectionPool.java:313)
at org.redisson.connection.pool.ConnectionPool.lambda$createConnection$3(ConnectionPool.java:279)
at org.redisson.misc.RedissonPromise.lambda$onComplete$0(RedissonPromise.java:183)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
at org.redisson.misc.RedissonPromise.tryFailure(RedissonPromise.java:96)
at org.redisson.client.RedisClient$2$1.run(RedisClient.java:242)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.redisson.client.RedisTimeoutException: Command execution timeout for command: (AUTH), params: (password masked), Redis client: [addr=redis://182.26.3.21:6379]
at org.redisson.client.RedisConnection.lambda$async$1(RedisConnection.java:251)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472)
... 2 common frames omitted
I can that it tries to connect to the master redis using redis://182.26.3.21:6379 instead of rediss://182.26.3.21:6379 which I gather is needed for tls connection.
I used ssl scan tool to verify that ssl 1.2 is enabled and here is the output.
D:\Users\03856u\Documents\sslScan>sslscan 182.26.3.21:6379
Version: 2.0.10 Windows 64-bit (Mingw)
OpenSSL 1.1.1e-dev xx XXX xxxx
Connected to 182.26.3.21
Testing SSL server 182.26.3.21 on port 6379 using SNI name 182.26.3.21
SSL/TLS Protocols:
SSLv2 disabled
SSLv3 disabled
TLSv1.0 disabled
TLSv1.1 disabled
TLSv1.2 enabled
TLSv1.3 enabled
TLS Fallback SCSV:
Server supports TLS Fallback SCSV
TLS renegotiation:
Session renegotiation not supported
TLS Compression:
Compression disabled
Heartbleed:
TLSv1.3 not vulnerable to heartbleed
TLSv1.2 not vulnerable to heartbleed
Supported Server Cipher(s):
Preferred TLSv1.3 128 bits TLS_AES_128_GCM_SHA256 Curve 25519 DHE 253
Accepted TLSv1.3 256 bits TLS_AES_256_GCM_SHA384 Curve 25519 DHE 253
Accepted TLSv1.3 256 bits TLS_CHACHA20_POLY1305_SHA256 Curve 25519 DHE 253
Accepted TLSv1.3 128 bits TLS_AES_128_CCM_SHA256 Curve 25519 DHE 253
Server Key Exchange Group(s):
TLSv1.3 128 bits secp256r1 (NIST P-256)
TLSv1.3 192 bits secp384r1 (NIST P-384)
TLSv1.3 260 bits secp521r1 (NIST P-521)
TLSv1.3 128 bits x25519
TLSv1.3 224 bits x448
TLSv1.2 128 bits secp256r1 (NIST P-256)
TLSv1.2 192 bits secp384r1 (NIST P-384)
TLSv1.2 260 bits secp521r1 (NIST P-521)
TLSv1.2 128 bits x25519
TLSv1.2 224 bits x448
SSL Certificate:
Signature Algorithm: sha256WithRSAEncryption
RSA Key Strength: 2048
Subject: redis.corp.com
Altnames: DNS:redis.corp.com, DNS:redis03.corp.com, DNS:redis01.corp.com, DNS:redis02.corp.com, IP Address:182.26.3.21, IP Address:182.26.3.54, IP Address:182.26.3.51
Issuer: UAT-CA
Not valid before: Oct 4 05:19:14 2021 GMT
Not valid after: Oct 3 05:19:14 2026 GMT
I have opened an issue in github and I was told that there was no issue with the library in connecting to sentinel on SSL, so I am guessing its an issue in my configuration. I looked at the documentation on github and couldn't figure it out. any help is welcome.
EDIT: so it seems that it was a bug with redisson, an issue was opened and a solution was proposed, and the creator created a fix and I tried it and it working. link to the github issue: https://github.com/redisson/redisson/issues/3950
