lettuce can't connect to docker redis

Question

I create a redis cluster with docker-compose and when I try to connect the cluster from my local machine with a java app, which also docker is located in, but lettuce gets timeout while jedis connects to cluster. Lettuce connects to main servers which are 7001 7002 7003 and after that it discover other nodes and again try to connect them and gets time out. Here is my code.

Docker-compose.yml

version: '3.8'

networks:
  redis-net:
    name: redis-net
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 10.0.0.0/16

services:

  redis-cluster:
    image: redis:latest
    container_name: redis-cluster
    command: redis-cli --cluster create 10.0.0.11:6379 10.0.0.12:6379 ... --cluster-replicas 1 --cluster-yes
    depends_on:
      - redis-1
      - redis-2
     ...
    networks:
      redis-net:
        ipv4_address: 10.0.0.2

  redis-1:
    build: ./redis
    container_name: redis-1
    image: redis-cluster-node
    networks:
      redis-net:
        ipv4_address: 10.0.0.11
    ports:
      - 7001:6379

  redis-2:
    image: redis-cluster-node
    container_name: redis-2
    networks:
      redis-net:
        ipv4_address: 10.0.0.12
    ports:
      - 7002:6379
    ...
      
  redis_commander:
    image: rediscommander/redis-commander:latest
    container_name: redis_web
    environment:
      REDIS_HOSTS: "local:redis-1:6379,local:redis-2:6379,..."
    ports:
      - "5000:8081"
    depends_on:
      - redis-1
      - redis-2
      ...
    networks:
      redis-net:
        ipv4_address: 10.0.0.3

Dockerfile

FROM redis:latest
COPY redis.conf /etc/redis/redis.conf
ENTRYPOINT ["redis-server","/etc/redis/redis.conf"]

redis.conf

port 6379
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000
appendonly yes
maxmemory-policy volatile-lru
maxmemory 3gb
bind 0.0.0.0

Test class

public static void lettuce() {
        RedisURI node1 = RedisURI.Builder.redis("localhost",7001).build();
        RedisURI node2 = RedisURI.Builder.redis("localhost",7002).build();
        RedisURI node3 = RedisURI.Builder.redis("localhost",7003).build();

        RedisClusterClient redisClient = RedisClusterClient.create(Arrays.asList(node1, node2, node3));
        ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder().dynamicRefreshSources(false).build();

        redisClient.setOptions(ClusterClientOptions.builder()
                .topologyRefreshOptions(topologyRefreshOptions)
                .build());
        StatefulRedisClusterConnection<String, String> connection = redisClient.connect();
        RedisAdvancedClusterCommands<String, String> syncCommands = connection.sync();
        syncCommands.set("test key", "test value");
}

Error

2022-02-12 11:06:11.494  WARN 21392 --- [ioEventLoop-6-4] i.l.core.cluster.RedisClusterClient      : connection timed out: /10.0.0.12:6379
2022-02-12 11:06:21.530  WARN 21392 --- [ioEventLoop-6-5] i.l.core.cluster.RedisClusterClient      : connection timed out: /10.0.0.11:6379
2022-02-12 11:06:31.564  WARN 21392 --- [ioEventLoop-6-6] i.l.core.cluster.RedisClusterClient      : connection timed out: /10.0.0.13:6379
2022-02-12 11:06:41.586  WARN 21392 --- [ioEventLoop-6-7] i.l.core.cluster.RedisClusterClient      : connection timed out: /10.0.0.14:6379
2022-02-12 11:06:51.625  WARN 21392 --- [ioEventLoop-6-8] i.l.core.cluster.RedisClusterClient      : connection timed out: /10.0.0.16:6379
2022-02-12 11:07:01.635  WARN 21392 --- [ioEventLoop-6-1] i.l.core.cluster.RedisClusterClient      : connection timed out: /10.0.0.15:6379
Exception in thread "restartedMain" java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect
    at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:94)
    at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:352)
    at io.lettuce.core.cluster.RedisClusterClient.connect(RedisClusterClient.java:400)
    at io.lettuce.core.cluster.RedisClusterClient.connect(RedisClusterClient.java:375)
    at com.exercise.redis.utils.RedisShardedPoolUtils.lettuce(RedisShardedPoolUtils.java:47)
    at com.exercise.redis.RedisExerciseApplication.main(RedisExerciseApplication.java:12)
    ... 5 more
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.15:6379
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:261)
    at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)

Thanks

I think what that error message is saying to me is that you're successfully connecting to one member of the cluster, but it's pointing you at another member of the cluster using the Docker-internal IP address. Can you run the application inside a container too? — David Maze, Feb 12 '22 at 10:51
(Consider deleting all of the `networks:` and `container_name:` blocks: Docker and Compose can assign network names, IP address, and container names just fine, and this will make your Compose file significantly shorter. For purposes of this question, do you get the same issue with a cluster of 2 replicas instead of 6? Editing the `docker-compose.yml` file to the minimum necessary to reproduce the issue will make the question a little more approachable.) — David Maze, Feb 12 '22 at 10:52
@DavidMaze hi, I tried what you say and I delete networks container_names and left only two node but when tried to make them a cluster, I get `Redis cluster requires at least 3 master nodes. At least 6 nodes are required.` error. After the error, I add other nodes and complete cluster to 6 nodes again and make them a cluster. In summary I only delete container_names and networks but it didn't work for me and got the same result. Now I will try to dockerize my spring app but in the end I don't want to reach redis cluster from inside of a docker — Can Şahintaş, Feb 12 '22 at 11:58
Editing the `docker-compose.yml` file in the question to remove the unnecessary settings and all but three nodes would definitely simplify things for other readers. — David Maze, Feb 12 '22 at 13:47

score 2 · Accepted Answer · answered Feb 19 '22 at 22:14

I found the solution with bitnami-redis here is my docker-compose.yml

services:
  redis-0:
    image: bitnami/redis-cluster
    container_name: redis-0
    restart: always
    networks:
      redis-net:
        ipv4_address: 172.22.0.100
    environment:
      - 'REDIS_PORT_NUMBER=7000'
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-0 redis-1 redis-2 redis-3 redis-4 redis-5'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=7000'
      - 'REDIS_CLUSTER_ANNOUNCE_IP=192.168.0.13' --local ip--
      - 'REDIS_CLUSTER_BUS_ANNOUNCE_PORT=17000'
      - 'REDIS_CLUSTER_DYNAMIC_IPS=no'
    ports:
      - "7000:7000"
      - "17000:17000"

  redis-1:
    image: bitnami/redis-cluster
    container_name: redis-1
    restart: always
    networks:
      redis-net:
        ipv4_address: 172.22.0.101
    environment:
      - 'REDIS_PORT_NUMBER=7001'
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-0 redis-1 redis-2 redis-3 redis-4 redis-5'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=7001'
      - 'REDIS_CLUSTER_ANNOUNCE_IP=192.168.0.13' --local ip--
      - 'REDIS_CLUSTER_BUS_ANNOUNCE_PORT=17001'
      - 'REDIS_CLUSTER_DYNAMIC_IPS=no'
    ports:
      - "7001:7001"
      - "17001:17001"

  redis-2:
    image: bitnami/redis-cluster
    container_name: redis-2
    restart: always
    networks:
      redis-net:
        ipv4_address: 172.22.0.102
    environment:
      - 'REDIS_PORT_NUMBER=7002'
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-0 redis-1 redis-2 redis-3 redis-4 redis-5'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=7002'
      - 'REDIS_CLUSTER_ANNOUNCE_IP=192.168.0.13' --local ip--
      - 'REDIS_CLUSTER_BUS_ANNOUNCE_PORT=17002'
      - 'REDIS_CLUSTER_DYNAMIC_IPS=no'
    ports:
      - "7002:7002"
      - "17002:17002"

  --- other nodes

  redis-cluster-init:
    image: redis:6.2
    container_name: redis-cluster-init
    restart: 'no'
    networks:
      redis-net:
        ipv4_address: 172.22.0.106
    depends_on:
      - redis-0
      - redis-1
      - redis-2
      - redis-3
      - redis-4
      - redis-5
    entrypoint: []
    command:
      - /bin/bash
      - -c
      - redis-cli --cluster create 172.22.0.100:7000 172.22.0.101:7001 172.22.0.102:7002 172.22.0.103:7003 172.22.0.104:7004 172.22.0.105:7005 --cluster-replicas 1 --cluster-yes

  redis_commander:
    image: rediscommander/redis-commander:latest
    container_name: redis_web
    environment:
      REDIS_HOSTS: "local:redis-0:7000,local:redis-1:7001,local:redis-2:7002,local:redis-3:7003,local:redis-4:7004,local:redis-5:7005"
    ports:
      - "5000:8081"
    depends_on:
      - redis-1
      - redis-2
      - redis-3
      - redis-4
      - redis-5
      - redis-0
      - redis-cluster-init
    networks:
      redis-net:
        ipv4_address: 172.22.0.107
networks:
  redis-net:
    name: redis-net
    driver: bridge
    ipam:
      config:
        - subnet: 172.22.0.0/16

Here is my lettuce code

RedisURI redis0 = RedisURI.create("127.0.0.1",7000);
RedisURI redis1 = RedisURI.create("127.0.0.1",7001);
RedisURI redis2 = RedisURI.create("127.0.0.1",7002);

RedisClusterClient clusterClient = RedisClusterClient.create(Arrays.asList(redis1,redis2,redis0));

StatefulRedisClusterConnection<String, String> connection = clusterClient.connect();

RedisAdvancedClusterCommands<String, String> syncCommands = connection.sync();

i get this error java.lang.UnsupportedOperationException with the above docker compose in lettuce, any idea why this happens? — Rani Selvaraj, Dec 22 '22 at 05:11

lettuce can't connect to docker redis

1 Answers1