0

I'm using kombu to consume rabbitmq messages from multiple queues. For any given message, is it possible to determine which queue delivered it?

nerd31415926535
  • 21
  • 3

2 Answers2

0

It is possible (and simple) to make distinctions based on the queue from which a message originated using kombu. You simply need to obtain the queue name via:

 print message.properties.get('user_id','no user id in message')

on the consumer's end. A more in-depth tutorial on this is available at https://adam.younglogic.com/2016/03/id-message-sender-kombu/

Philip DiSarro
  • 1,007
  • 6
  • 9
0

As far as I can tell, there is no direct way to access the name of the queue on which a message is received in kombu. The proposed solution by Philip isn't sufficiently generic and strangely relies on a user_id being specified in the publish call. This presents two problems:

  • Not every producer may send user_id information along with a message. In our test case, user_id was never set by default. Additionally, it would be no more advantageous to use the user_id field than simply encoding the queue name in the message itself.
  • The benefit of using routing keys when we publish messages is that, from the publisher's perspective, we don't need to care where the message goes. If we encode queue names (in any way, user_id or otherwise) in the published messages, then it defeats the purpose of routing.

We identified two possible solutions when using the ConsumerMixin:

  • Setting the kombu consumer_tag field on Consumer objects, then defining Consumers to consume on only a single queue. If you need to listen on multiple queues, define multiple Consumers. This attribute is less than ideal because it's not documented and involves string matching, as the field is appended with an integer.

  • Using Python partial functions to wrap the Consumer callback function and pass the queue name:

    con = Consumer(queue=queue, callback=[partial(self.callback, queue_name=queue.name)])
...
def callback(self, body, message, queue_name):

Neither of these solutions is particularly elegant. Better would be if kombu would simply include a reference to the queue on which a message was received along with the Message sent to the Consumer callback.

Zack Panitzke
  • 1