I am using Qpid Proton using Python to consume messages from a queue in AMQ. I am learning this as I go - I am far from an expert at it so I am reaching out for some help. I have a situation where the AMQ queue holds messages that represent work to be done and it goes against a database (which takes time to do). So I am Receiving "N-number" of messages from the queue, doing the work, and repeat. The problem is I cant' figure out a way to know if the queue is empty...
Is there a way to detect if a queue in a broker is empty after the last message has been consumed? If this were a single-threaded consumer, then no problem; but I may have several consumers going against the one queue so knowing exactly how many messages are on the queue is not possible.
So for example, on the on_message method (this comes from the Qpid Python documentation (direct_recv.py) we have this:
def on_message(self, event):
if event.message.id and event.message.id < self.received:
# ignore duplicate message
return
if self.expected == 0 or self.received < self.expected:
print(event.message.body)
self.received += 1
if self.received == self.expected:
event.receiver.close()
event.connection.close()
self.acceptor.close()
This fragment shows that when self.received == self.expected, then the consumer will close. For example, suppose a queue in a broker has 65 messages and the consumer calling this method sets self.expected = 25, then after 25 messages are consumed then this will end; I can make that work.
Now suppose the code tries to consume the same 25 messages (self.expected = 25), however the queue only has at that moment 12 messages then, the receiver will hang indefinitely - until more messages are added to the queue. However in my case, no more messages will be added.
Is there a way, to query the queue and determine if the queue is "empty" and if so, exit? In other words something like:
if ( self.received == self.expected ) or (empty queue)
event.receiver.close()
event.connection.close()
self.acceptor.close()
I just can't figure out how to know if the queue is empty. Is this even possible?
As a side note I am able to make this work by using a BlockingConnection. It is not as elegant as using the Receive class and is also not as fast. So essentially:
- create the Blocking Connection
- write a loop to receive and accept "N-number" of messages (or wait for a timeout when the queue is empty)
- close the Blocking Connection.
Any help would be appreciated, thanks!