Kombu+RabbitMQ: Check whether a queue is empty

Question

Architecture

Consider a system with DB records. Each record can be in a live or expired status; live records should be processed periodically using an external software module.

I have solved this using a classic producer - consumer architecture with Kombu and RabbitMQ. The producer fetches the records from the DB every few seconds, and the consumer handles them.

The problem

The number of live events greatly varies, and on peak hours the consumer can't handle the load and the queue is clogged with thousand of items.

I would like to make the system adaptive, so that the producer will not send new events to the consumer if the queue is empty.

What have I tried

Searching the Kombu documentation / API
Inspecting the Queue object
Using the RabbitMQ REST API: http://<host>:<port/api/queues/<vhost>/<queue_name>. It works, but it's yet another mechanism to maintain, and I prefer an elegant solution within Kombu.

How do I check whether a RabbitMQ is empty using Python's Kombu?

odedfos · Answer 1 · 2018-02-05T14:05:17.733

2

You can call queue_declare() on the kombu Queue object.

According to the docs the function returns:

Returns a tuple containing 3 items:
        the name of the queue (essential for automatically-named queues)
        message count
        consumer count

Therefore you can do:

name, msg_count, consumer_count = queue.queue_declare()

edited Feb 05 '18 at 14:05

answered Feb 01 '17 at 19:58

odedfos

4,491
3
30
42

How to import queue ? – Kishan Mehta Apr 21 '17 at 04:04
1

from kombu import Queue. queue is a Queue object. – odedfos Apr 23 '17 at 06:27
Could you please reference the documentation with a link as I can't find this in the documentation anymore. – RexFuzzle Feb 05 '18 at 12:21
You can find it in the docstring of the amqp module in channel.py https://github.com/celery/py-amqp/blob/master/amqp/channel.py#L1137 – odedfos Feb 05 '18 at 14:03
what if i have multiple queue in one server? – Devang Hingu Apr 15 '22 at 09:17

Kombu+RabbitMQ: Check whether a queue is empty

Architecture

The problem

What have I tried

1 Answers1