Are RabbitMQ messages lost if published to an exchange with a specific routing key when the consumer queue doesn't exist yet?

Question

When I publish a message to RabbitMQ, is the message sent to the exchange with a specific routing key, and when the consumer queue doesn't exist yet, will the message be lost?

Answer 1

Yes, RabbitMQ will drop the message.

But, you care about your data, so you're using publisher confirmations, correct? If so, RabbitMQ will send a message back indicating that your message was not routed.

You can also specify an alternate exchange to catch these messages.

---

_{NOTE: Team RabbitMQ monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.}

Answer 2

The default configuration in almost each MQ-System is pub/sub.

A publisher sends a message on a topic. The message gets forwarded to all currently active subscribed subscribers listening on the topic.

B --> subscribe to MQ topic

A ---pub@topic---> B_any

In this setting the message won't get redelivered to inactive subscribers. Aka the message might get lost with pub/sub.

2)

Another aproach is a client-side improvement called request-reply pattern. The publisher combines the message with a temporary "backward-reply-topic", any subscriber that got the message will send a reply that the message was received. Normally the library doesn't care about how many subscribers received the message but at least one replied. That technique is called request-reply. (If the library indicated a no-reply-error you need to resend the message obviously.)

B --> subscribe to MQ topic

A ---req@topic---> B_any
A <-----reply----- B_any

In your case that would help if the receiving end is the only one that matches the topic. With this the sender can check that the message was received.

3)

If you want to guarantee that the message get delivered to an unknown amount of receivers, even if they are offline at a later time, this won't work out of the box.

To achieve this anyways you need to turn on persistence features in your MQ System. The persistence features use streaming and/or named durable consumer patterns. With persistence features you can achieve many more guarantees and remove the check and redelivery-on-failure function from your sender. The MQ System should store the message and handle that it gets received and consumed correctly. Have a look at the "Feature Matrix" here: https://www.rabbitmq.com/streams.html

The "per message persistance retention policy" would be like a work queue and just a at least one receiver guarantee. Kind of like the request-reply pattern, but you don't need to implement a resend method, since the MQ system will do it.

If traffic is not an issue and you know exactly how many consumers you have, you might give your consumers unique topics and send the message multiple times.

For multiple consumers that you do not know (at the publisher side) you need other retention policies to keep guarantees across all. See https://www.rabbitmq.com/streams.html#retention - you could set your stream to keep messages for a given time period and tell your consumers to start reading messages from an old time in point, for example when they went offline... Hope you find what you need.

A ---req@topic---> MQ Stream storage 
A <-----reply----- MQ Stream storage 
and
(async)  MQ Stream storage ---req@topic---> B
(async)  MQ Stream storage <-----reply----- B

Sometimes you can even specify if B should be a pull or push based consumer in that setting, but that depends on the MQ System. I don't know if RabbitMQ has that.

TLDR: Check out this answer RabbitMQ: In pub/sub is the consumer polling the queue for new messages or does the server push messages?

For RabbitMQ persistance with streaming and consumers read through this and this

Also i am not entirly sure, but I think that RabbitMQ uses the wording "pub/sub with consumers" for basic per message persistance. Double check if you use "pub/sub with (volatile) subscriber" or "pub/sub with (persistent) consumers".

---

This behaviour is the same for a lot of modern MQ-Systems but with slightly different wording.