0

Hi I used official docker-compose file to setup my redis with sentinel as below:

version: '2'

networks:
  app-tier:
    driver: bridge

services:
  redis:
    image: 'bitnami/redis:latest'
    environment:
      - REDIS_REPLICATION_MODE=master
      - REDIS_PASSWORD=str0ng_passw0rd
    networks:
      - app-tier
    ports:
      - '6379'
  redis-slave:
    image: 'bitnami/redis:latest'
    environment:
      - REDIS_REPLICATION_MODE=slave
      - REDIS_MASTER_HOST=redis
      - REDIS_MASTER_PASSWORD=str0ng_passw0rd
      - REDIS_PASSWORD=str0ng_passw0rd
    ports:
      - '6379'
    depends_on:
      - redis
    networks:
      - app-tier
  redis-sentinel:
    image: 'bitnami/redis-sentinel:latest'
    environment:
      - REDIS_MASTER_PASSWORD=str0ng_passw0rd
    depends_on:
      - redis
      - redis-slave
    ports:
      - '26379-26381:26379'
    networks:
      - app-tier

Then I set my sentinel URL in celery as below:

REDIS_URL = "sentinel://192.168.5.80:26381"
print("REDIS_URL", REDIS_URL)
app = celery.Celery('tasker')
app.conf.broker_url = REDIS_URL
app.conf.broker_transport_options = { 'master_name': 'mymaster'}}

but somehow I am getting below error after some time:

[2023-05-11 21:03:35,301: CRITICAL/MainProcess] Unrecoverable error: TypeError('Channel._connparams.<locals>.Connection.disconnect() takes 1 positional argument but 2 were given')
Traceback (most recent call last):
  File "/home/worker/.local/lib/python3.11/site-packages/kombu/transport/virtual/base.py", line 925, in create_channel
    return self._avail_channels.pop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: pop from empty list

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/worker/.local/lib/python3.11/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
           ^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/redis/connection.py", line 610, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
            ^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/redis/connection.py", line 675, in _connect
    raise err
  File "/home/worker/.local/lib/python3.11/site-packages/redis/connection.py", line 663, in _connect
    sock.connect(socket_address)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/worker/.local/lib/python3.11/site-packages/celery/worker/worker.py", line 203, in start
    self.blueprint.start(self)
  File "/home/worker/.local/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start
    step.start(parent)
  File "/home/worker/.local/lib/python3.11/site-packages/celery/bootsteps.py", line 365, in start
    return self.obj.start()
           ^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 332, in start
    blueprint.start(self)
  File "/home/worker/.local/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start
    step.start(parent)
  File "/home/worker/.local/lib/python3.11/site-packages/celery/worker/consumer/connection.py", line 21, in start
    c.connection = c.connect()
                   ^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 428, in connect
    conn = self.connection_for_read(heartbeat=self.amqheartbeat)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 434, in connection_for_read
    return self.ensure_connected(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 460, in ensure_connected
    conn = conn.ensure_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/kombu/connection.py", line 381, in ensure_connection
    self._ensure_connection(*args, **kwargs)
  File "/home/worker/.local/lib/python3.11/site-packages/kombu/connection.py", line 433, in _ensure_connection
    return retry_over_time(
           ^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.11/site-packages/kombu/utils/functional.py", line 312, in retry_over_time
    return fun(*args, **kwargs)

  File "/home/worker/.local/lib/python3.11/site-packages/redis/retry.py", line 49, in call_with_retry
    fail(error)
  File "/home/worker/.local/lib/python3.11/site-packages/redis/connection.py", line 610, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
                                           ^^^^^^^^^^^^^^^^^^^^^^
TypeError: Channel._connparams.<locals>.Connection.disconnect() takes 1 positional argument but 2 were given

Can you tell me where I am making mistake? I even tried to pass password in URL or as below but it did not work.

app.conf.broker_transport_options = { 'master_name': 'mymaster' , 'sentinel_kwargs': {'password': redis_password}}

FYI: I am using latest version of celery and redis in requirements and I had to remove some of the traceback as stackoverflow detects it as a spam

hmffa
  • 5
  • 3

1 Answers1

0

Connection Details: Make sure you have the correct host and port details in your Flask application. Remember that Redis Sentinel runs on different ports (typically 26379, as per your Docker Compose file) than a normal Redis instance (which runs on 6379). Make sure your Flask application is trying to connect to the right port.

Network Access: If your Flask application is running inside a Docker container, make sure it's on the same Docker network as your Redis Sentinel service. If they're on different networks, they won't be able to communicate. According to your Docker Compose file, your Redis services are running on the 'app-tier' network. Your Flask application should also be on this network.

Firewall Rules: If your Flask application is running on a different machine, make sure there are no firewall rules preventing it from connecting to the Redis Sentinel port.

Use the Right Client: Make sure you're using a Redis client in your Flask application that supports Sentinel. redis-py is a popular Python client for Redis that supports Sentinel. Here's a sample code to connect to Redis through Sentinel using redis-py: enter image description here

In this example, replace 'localhost' with your Redis Sentinel host, and 'mymaster' with your Redis master name. Check Redis Sentinel Logs: Lastly, check the logs of your Redis Sentinel container for any errors or warnings. You can view the logs by running docker logs <container_id> in your terminal.

Remember to update the sensitive information like host, port, and passwords according to your setup. Hope this helps, and let me know if you have more questions!

psssDev
  • 46
  • 2