2

I have the following use case that I'm trying to setup in rabbit MQ:

  1. Normally process A should handle all messages sent to queue A.
  2. However if process A goes down (is no longer consuming from queue A) Then process B should handle the messages until process A comes back up.

At first it looks like consumer priorities might be the solution. https://www.rabbitmq.com/consumer-priority.html. However that will send messages to process B when process A is just blocked working on other messages. I only want them sent to process B when process A is down.

A 2nd option might be dead lettering. https://www.rabbitmq.com/dlx.html. If process A is not reading from queue A the messages will eventually time out and then move to an exchange that forwards them to a queue that process B reads. However that options requires waiting for the message to timeout which is not ideal. Also the message could timeout even while process A is still working which is not ideal.

Any ideas how rabbit MQ could be configured for the use case described above? Thanks

innominate227
  • 11,109
  • 1
  • 17
  • 20
  • What is your use case? Do process A and B compute different stuff? Is it an option to have another instance of process A to handle messages ? Do you really need to have all messages handled immediately (no timeout)? – cdelmas Jan 02 '19 at 09:39
  • @cdelmas process A is likely to have data cached that will allow it to respond to messages from queue A significantly (100x) faster than process B. The processes are identical (other than configuration that tells them which queue to have affinity for). Clients are waiting on a response so as fast as possible is desired. – innominate227 Jan 02 '19 at 18:40
  • Ok, so next question: can your client wait for few seconds for a response, or do they require quick responses? Do their request have a timed validity? – cdelmas Jan 03 '19 at 13:06
  • A few seconds is a reasonable wait time. There is no timed validity on the requests. – innominate227 Jan 03 '19 at 20:23

1 Answers1

0

According to your answers to my questions, I would probably use a priority on consumer so that process A handles a maximum of messages, along with a high prefetch count (if possible, and you must ensure your process can handle such a high number).

Then, process B would handle the messages that process A cannot handle due to the high load, or all the messages when process A is not available. It is probably acceptable that in the case of high load some messages are handled with a higher delay. Do not forget to set a low prefetch count for process B.

Hope this helps.

cdelmas
  • 840
  • 8
  • 15