Spring Boot Caching with Redis Sentinel always connects to master node

Question

I have a Spring Boot (2.3.1.RELEASE) app that has caching with Redis Sentinel. This is my configuration for the Sentinel connection:

@Bean
public LettuceConnectionFactory redisConnectionFactory() {
   RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
            .master(redisProperties.getSentinel().getMaster());
    redisProperties.getSentinel().getNodes().forEach(s -> sentinelConfig.sentinel(s, redisProperties.getPort()));
    sentinelConfig.setPassword(RedisPassword.of(redisProperties.getPassword()));
    return new LettuceConnectionFactory(sentinelConfig);
}

And this is my caching manager configuration:

@Bean
public RedisCacheManager cacheManager() {
    Map<String, RedisCacheConfiguration> cacheConfigs = new HashMap<>();
    cacheConfigs.put("cache1", RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(ttlMinutes)));
    cacheConfigs.put("cache2", RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(ttlMinutes)));

    return RedisCacheManager.builder(redisConnectionFactory())
            .cacheDefaults(RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(ttlMinutes)))
            .withInitialCacheConfigurations(cacheConfigs)
            .transactionAware()
            .build();
}

Everything works fine from caching perspective.

However, If I turn on debug logs inside the io.lettuce.core.protocol.CommandHandler, I see that it is always connecting to the same node (master). Which I can confirm by looking at the logs on the node.

Everywhere I look online, this seems like the correct configuration.

This brings me to my question:

Is it possible to configure the Spring caching abstraction to use the master node only for writes and the slave nodes for reads?

Is this expectation even valid? Or this is the way Sentinel is supposed to be used (all requests go to master)?

Anish B. · Accepted Answer · 2020-11-19T05:01:09.063

Yes, it can be done.

From Spring Data Redis Docs - 10.4.4. Write to Master, Read from Replica :

It is said that Spring Data Redis provides a Redis Master/Replica setup which not only allows data to be safely stored at more nodes but also allows reading data from replicas while pushing writes to the master by using Lettuce.

For this, you have to update redisConnectionFactory() method in the configuration class :

 @Bean
 public LettuceConnectionFactory redisConnectionFactory() {
   LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
            .readFrom(ReadFrom.REPLICA_PREFERRED)
            .build();
   RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
            .master(redisProperties.getSentinel().getMaster());
   redisProperties.getSentinel().getNodes().forEach(s -> sentinelConfig.sentinel(s, redisProperties.getPort()));
   sentinelConfig.setPassword(RedisPassword.of(redisProperties.getPassword()));
   return new LettuceConnectionFactory(sentinelConfig, clientConfig);
}

score 0 · Answer 2 · answered Nov 18 '20 at 10:53

if you want to write master and read only from all slaves(replicas), you can use below configuration.

These configurations include connection pooling,write to master and read from all slave nodes with round-robin load balancing.

    @Bean
    public RedisConnectionFactory lettuceConnectionFactory(RedisProperties redisProperties, LettucePoolingClientConfiguration lettucePoolingClientConfiguration) {
        final RedisSentinelConfiguration redisSentinelConfiguration = new RedisSentinelConfiguration().master(redisProperties.getMaster());
        redisSentinelConfiguration.setDatabase(redisProperties.getDbIndex());
        addSentinels(redisProperties, redisSentinelConfiguration);
        return new LettuceConnectionFactory(redisSentinelConfiguration, lettucePoolingClientConfiguration);
    }

    private void addSentinels(RedisProperties redisProperties, RedisSentinelConfiguration redisSentinelConfiguration) {
        redisProperties.getNodes()
                .forEach(node -> {
                    final String[] splitted = node.split(NODE_SPLITTER);
                    final String host = splitted[0];
                    final int port = Integer.parseInt(splitted[1]);
                    redisSentinelConfiguration.addSentinel(RedisNode.newRedisNode()
                            .listeningAt(host, port)
                            .build());
                });
    }

    @Bean
    public LettucePoolingClientConfiguration lettucePoolingClientConfiguration(ClientOptions clientOptions, ClientResources clientResources, RedisProperties redisProperties) {
        return LettucePoolingClientConfiguration.builder()
                .readFrom(ReadFrom.ANY_REPLICA)
                .poolConfig(genericObjectPoolConfig(redisProperties))
                .clientOptions(clientOptions)
                .clientResources(clientResources)
                .build();
    }

    @Bean
    public GenericObjectPoolConfig genericObjectPoolConfig(RedisProperties redisProperties) {
        final GenericObjectPoolConfig config = new GenericObjectPoolConfig();
        config.setMaxIdle(redisProperties.getPoolMaxIdle());
        config.setMinIdle(redisProperties.getPoolMinIdle());
        config.setMaxTotal(redisProperties.getPoolMaxTotal());
        config.setBlockWhenExhausted(false);
        config.setMaxWaitMillis(redisProperties.getPoolMaxWaitMillis());
        return config;
    }

    @Bean
    public ClientOptions clientOptions(RedisProperties redisProperties) {
        return ClientOptions.builder()
                .autoReconnect(true)
                .disconnectedBehavior(ClientOptions.DisconnectedBehavior.REJECT_COMMANDS)
                .timeoutOptions(TimeoutOptions.builder().fixedTimeout(Duration.ofSeconds(redisProperties.getCommandTimedOutSec())).build())
                .build();
    }

    @Bean(destroyMethod = "shutdown")
    public ClientResources clientResources() {
        return DefaultClientResources.create();
    }

you should import new lettuce core version for load balance read mode(ReadFrom.ANY_REPLICA), it will come with spring-boot 2.4.0

pom.xml

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>io.lettuce</groupId>
                    <artifactId>lettuce-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>io.lettuce</groupId>
            <artifactId>lettuce-core</artifactId>
            <version>6.0.1.RELEASE</version>
    </dependency>

Spring Boot Caching with Redis Sentinel always connects to master node

2 Answers2