Disable global qos in rabbitmq from celery

Question

I have created a RabbitMQ 3-node cluster using three aws ec2 servers. I'm trying to access the quorum queue I created using celery. When I connect it gives the error

raise error_for_code(reply_code, reply_text,
amqp.exceptions.AMQPNotImplementedError: Basic.consume: (540) NOT_IMPLEMENTED - queue 'Replica_que' in vhost '/' does not support global qos

I suppose it will work if I disabled global qos but I couldn't find where I can do it. How do I disable global qos in celery?

my celery code

from celery import Celery
from time import sleep
import kombu


broker_uri=['amqp://xxxx:5672/', 'amqp://xxxx:5672/','amqp://xxx:5672/']
backend_uri="mongodb+srv://xxxxx"

app = Celery('TestApp', broker=broker_uri,backend=backend_uri)

app.config_from_object('celeryconfig')
app.conf.task_default_exchange='Replica_que'
app.conf.task_default_routing_key='Replica'

@app.task
def reverse(text):
    sleep(10)
    return text[:-1]

and the config code

from kombu import Queue

task_queues = [Queue(name="Replica_que", queue_arguments={"x-queue-type": "quorum"})]

task_routes = {
    'tasks.add': 'Replica_que',
}

score 1 · Answer 1 · answered Sep 21 '21 at 12:30

This was possible by adding a celeryconfig.py file,

from kombu import Queue

task_queues = [Queue(name="Replica_que", queue_arguments={"x-queue-type": "quorum"})]

task_routes = {
    'tasks.add': 'Replica_que',
}

and creating a custom QoS class: https://github.com/celery/celery/issues/6067

So I added the QoS class

class NoChannelGlobalQoS(bootsteps.StartStopStep):
    requires = {'celery.worker.consumer.tasks:Tasks'}

    def start(self, c):
        qos_global = False

        c.connection.default_channel.basic_qos(0, c.initial_prefetch_count, qos_global)
        def set_prefetch_count(prefetch_count):
            return c.task_consumer.qos(
                prefetch_count=prefetch_count,
                apply_global=qos_global,
            )
        c.qos = QoS(set_prefetch_count, c.initial_prefetch_count)

app.steps['consumer'].add(NoChannelGlobalQoS)

Currently this is an issue in celery related to quorum queue but this works.

Disable global qos in rabbitmq from celery

1 Answers1