Published and waiting for response on a topic with RabbitMQ / EasyNetQ - how to get only yours?

Question

When a publisher expects an answer to a message, how to ensure it will get only relevant answers (to its own messages) when you scale it out?

We have a client process that publishes a message for a server process to answer. Additionally, we have a "listener" process that just needs to consume both the questions and the answers without publishing anything. Also, the server process might be broken to several ones on the future, creating a message cascade. We can't use the request/response, since we need the listener and then again, when we will have the cascade... Besides, we will have several question/answer categories, and request/response in EasyNetQ doesn't support topics.

Our solution with EasyNetQ was simple topic-based publish/subscribe: client publishes to "question" topic, subscribes to "answer", server subscribes to "question" and publishes to "answer", and the listener just subscribes to both.

The problem is when you scale out the client. Two instances of it now publish questions, but since they are both subscribed to a single "answer" topic, one might get the answer to the question published by the other instance, and not get his own.

The solution we found is to have the client use a uniquely named queue when subscribing to "answer" - this way each client will get all of the answers, and just needs to ignore the ones that are not his. However, this solution has some performance drawbacks and also results in uniquely named queues accumulating in RabbitMQ each time a client crashes (or just restarted during development, etc.).

Client, sending an object msg:

string corrId = Guid.NewGuid().ToString();

// Register the corrId in a dictionary
//...

var myMessage = new MyMessage {correlationId =corrId, realMessage = msg};
easyNetQBus.Subscribe<MyMessage>("mqClient"+uniqueSuffix, HandleMsg, x => x.WithTopic("answer"));
easyNetQBus.Publish(myMessage, "question");

// In HandleMsg, we see if we have issued questions with the correlation id that came with the answer (lookup in the dictionary) and if not, ignore it

Server:

easyNetQBus.Subscribe<MyMessage>("mqServer", HandleMsg, x => x.WithTopic("question"));

// In HandleMsg, we publish the answer back to "answer" with the correlation id from the question

Is there another pattern we should be using? We could put inside each message a unique topic/queue to send the answer to, but this complicates the lives of the listener and the flexibility of the future participants in the cascade I mentioned...

Answer 1

One way to solve your problem would be to use the Remote Procedure Call pattern. This provides an easy, scalable way to give each client a unique queue.

From the example:

• When the Client starts up, it creates an anonymous exclusive callback queue. For an RPC request, the Client sends a message with two properties: ReplyTo, which is set to the callback queue and CorrelationId, which is set to a unique value for every request.
• The request is sent to an rpc_queue queue. The RPC worker (aka: server) is waiting for requests on that queue. When a request appears, it does the job and sends a message with the result back to the Client, using the queue from the ReplyTo property.
• The client waits for data on the callback queue. When a message appears, it checks the CorrelationId property. If it matches the value from the request it returns the response to the application.

If your callback queues are auto-deleting, exclusive queues (RabbitMQ queue documentation), then they should not pile up when either the clients or the server restart.

With some workloads queues are supposed to be short lived. While clients can delete the queues they declare before disconnection, this is not always convenient. On top of that, client connections can fail, potentially leaving unused resources (queues) behind.

There are three ways to make queue deleted automatically:

• Exclusive queues (covered below)
• TTLs (also covered below)
• Auto-delete queues

An auto-delete queue will be deleted when its last consumer is cancelled (e.g. using the basic.cancel in AMQP 0-9-1) or gone (closed channel or connection, or lost TCP connection with the server).

If a queue never had any consumers, for instance, when all consumption happens using the basic.get method (the "pull" API), it won't be automatically deleted. For such cases, use exclusive queues or queue TTL. (RabbitMQ queue documentation)