How can I connect Flower to the broker RabbitMQ in docker-compose

Question

How I can connect Flower to RabbitMQ. My celery running tasks do not appear on Flower. Although they are running fine. On the Flower UI, when I click on "broker", I get this error message on the terminal [E 210326 10:53:07 broker:78] RabbitMQ management API call failed: Timeout while connecting

This is my docker-compose file:

version: "3.8"

services:
    base:
        container_name: base
        image: postgres:13
        ports:
            - "5433:5432"
        environment:
            - POSTGRES_USER=${POSTGRES_USER}
            - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
            - POSTGRES_DB=${POSTGRES_DB}
        volumes:
            - /var/lib/postgresql/data
    api:
        build:
            context: ./api/
            dockerfile: Dockerfile
        volumes:
            - ./api/app:/app/app
        ports:
            - "1000:80"
        depends_on:
            - base
        env_file:
            - .env
        command: ["/start-reload.sh"]

    frontend:
        build:
            context: ./frontend/
            dockerfile: Dockerfile
        ports:
            - "3000:3000"
        volumes:
            - /app/node_modules
            - ./frontend:/app
        environment:
            - NODE_ENV=development
        stdin_open: true
        links:
            - api

    rabbitmq:
        image: "bitnami/rabbitmq:latest"
        ports:
            - "4369:4369"
            - "5672:5672"
            - "25672:25672"
            - "15672:15672"
        volumes:
            # - "rabbitmq_data:/bitnami"
            - /bitnami

    redis:
        image: "bitnami/redis:latest"
        environment:
            - REDIS_PASSWORD=password123
        ports:
            - "6379:6379"
        volumes:
            # - "redis_data:/bitnami/redis/data"
            - /bitnami/redis/data

    flower:
        image: mher/flower
        command: ["flower", "--broker=amqp://user:bitnami@rabbitmq:5672//", "--port=8888"]
        ports:
            - 8888:8888
        depends_on:
            - rabbitmq
            - redis

    worker:
        build:
            context: ./api/
            dockerfile: Dockerfile.celery
        depends_on:
            - rabbitmq
            - redis
        volumes:
            - ./api/app:/app
        command: celery -A app.worker.celery_worker worker --loglevel=INFO --concurrency=4 -Q test-queue

score 3 · Answer 1 · answered Jun 06 '21 at 18:34

3

I have opened an issue in Flower project and will attempt to fix this if I have spare time but in the mean time in the investigation section you should see the reason for this.

Issue in github: https://github.com/mher/flower/issues/1112

answered Jun 06 '21 at 18:34

Tomasz Kluczkowski

145
1
11

J. Choi · Answer 2 · 2021-03-28T09:12:19.193

2

I also suffered from the same problem in my tests of last three days. Flower correctly worked only once during the tests. And I failed to identify why it worked at that time.

For me, the only method to solve this problem was downgrading the docker image from mher/flower:latest or mher/flower:0.9.7 to mher/flower:0.9.5. Changing command line options, environment variables, and docker-compose.yml contents did not help. Everything works fine after the downgrade.

Note. My environment was

Machine - Windows 10 with WSL2
Main - ubuntu:20.04 with Django 3.1.7, celery[librabbitmq,redis] 5.0.5, django-celery-beat 2.2.0
Celery Broker - rabbitmq:management
Celery ResultBackend - redis:latest
Celery Flower - mher/flower:latest > mher/flower:0.9.5

edited Mar 28 '21 at 09:12

answered Mar 28 '21 at 08:54

J. Choi

1,616
12
23

Thank you ! I was struggling with the last version of the container and now everything works perfectly. I think there is a a mismatch between code, code documentation and container documentation. I will investigate if I find some time. – Julien Tourille May 07 '21 at 16:10
I am using the latest version with environment variable answer from Jhkcia and it is working fine. No need to downgrade. – klement omeri Nov 15 '21 at 11:33
@klement omeri Hi. This problem seems like to be unsolved untill nowadays. In my opinion, the answer of Tomasz Kluczkowski should be the accepted one. Anyway, I agree that the downgrading cannot be a good solution. – J. Choi Nov 16 '21 at 10:50

score 1 · Answer 3 · answered Jul 09 '21 at 14:47

You can use this docker-compose file:

version: '3.4'
services:
    rabbitmq:
        image: rabbitmq
        ports: 
          - 5672:5672


    flower:
        image: mher/flower
        environment: 
            - CELERY_BROKER_URL=pyamqp://guest@rabbitmq//
            - FLOWER_PORT=5555
        ports: 
            - 5555:5555
        depends_on: 
            - rabbitmq

*** remember not to use single quotation for CELERY_BROKER_URL value!! the below config will raise error:

        - CELERY_BROKER_URL='pyamqp://guest@rabbitmq//'

This solved my problem, it should be the accepted answer. Thanks! — klement omeri, Nov 15 '21 at 11:32

score 1 · Answer 4 · edited Nov 23 '21 at 17:20

1

Try to connect to your RabbitMQ by Docker IP.

Example command:

["flower", "--broker=amqp://user:bitnami@172.25.0.2:5672//", "--port=8888"]

edited Nov 23 '21 at 17:20

ouflak

2,458
10
44
49

answered Nov 18 '21 at 20:31

gustavopoa

11
1

score -1 · Answer 5 · answered Mar 26 '21 at 11:19

You need to add network to combine them in same network. After that you can use service names as hostname. Like rabbitmq

version: "3.8"

services:
    base:
        container_name: base
        image: postgres:13
        ports:
            - "5433:5432"
        environment:
            - POSTGRES_USER=${POSTGRES_USER}
            - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
            - POSTGRES_DB=${POSTGRES_DB}
        volumes:
            - /var/lib/postgresql/data
        networks:
            - app-network
    api:
        build:
            context: ./api/
            dockerfile: Dockerfile
        volumes:
            - ./api/app:/app/app
        ports:
            - "1000:80"
        depends_on:
            - base
        env_file:
            - .env
        command: ["/start-reload.sh"]
        networks:
            - app-network
    frontend:
        build:
            context: ./frontend/
            dockerfile: Dockerfile
        ports:
            - "3000:3000"
        volumes:
            - /app/node_modules
            - ./frontend:/app
        environment:
            - NODE_ENV=development
        stdin_open: true
        links:
            - api
        networks:
            - app-network    
    rabbitmq:
        image: "bitnami/rabbitmq:latest"
        ports:
            - "4369:4369"
            - "5672:5672"
            - "25672:25672"
            - "15672:15672"
        volumes:
            # - "rabbitmq_data:/bitnami"
            - /bitnami
        networks:
            - app-network    
    redis:
        image: "bitnami/redis:latest"
        environment:
            - REDIS_PASSWORD=password123
        ports:
            - "6379:6379"
        volumes:
            # - "redis_data:/bitnami/redis/data"
            - /bitnami/redis/data
        networks:
            - app-network    
    flower:
        image: mher/flower
        command: ["flower", "--broker=amqp://user:bitnami@rabbitmq:5672//", "--port=8888"]
        ports:
            - 8888:8888
        depends_on:
            - rabbitmq
            - redis
        networks:
            - app-network    
    worker:
        build:
            context: ./api/
            dockerfile: Dockerfile.celery
        depends_on:
            - rabbitmq
            - redis
        volumes:
            - ./api/app:/app
        command: celery -A app.worker.celery_worker worker --loglevel=INFO --concurrency=4 -Q test-queue
        networks:
            - app-network    
    networks:
      app-network:
        driver: bridge

docker-compose automatically creates a default network for all the services. I checked networks. It is already there. — Sami Al-Subhi, Mar 26 '21 at 11:59

How can I connect Flower to the broker RabbitMQ in docker-compose

5 Answers5