2

I am trying to understand how rabbitmq per-connection flow-control works with multiple consumers. In particular what would happen if one consumer were to hang? Would flow control be invoked and how would it affect the rest of the consumers? Would the behaviour depend upon whether the queues were durable or autodeleting?

Thanks.

C R
  • 2,182
  • 5
  • 32
  • 41
  • Please describe what you mean by "flow control" as that can have several meanings in RabbitMQ depending on your setup. – theMayer Feb 01 '14 at 03:16
  • I mean per-connection. I changed the question and added a link to the appropriate rabbitmq documentation page – C R Feb 01 '14 at 14:38

2 Answers2

2

Rabbit MQ uses "Credit Flow Control".

Essentially, whenever a message is received on a channel a credit is deducted. Credit starts at a default level, e.g. 200, and when it dips below 0, connections are blocked. After a certain number of messages are consumed and ACKed, the credit is bumped up a certain amount.

You can read more about it here:

http://videlalvaro.github.io/2013/09/rabbitmq-internals-credit-flow-for-erlang-processes.html

Rene Wooller
  • 1,107
  • 13
  • 22
1

Per-connection flow control describes what happens when a publisher (or group of publishers) is sending messages to queues faster than the queues are being processed. This is a safety feature as RabbitMQ becomes unstable at some point when the queue fills without bound. From the documentation, this is automatic:

RabbitMQ will block connections which are publishing too quickly for queues to keep up. No configuration is required.

Unfortunately, the documentation is not terribly specific on when/how this flow control is implemented, other than "several times per second." So, if one consumer gets stuck, as long as the other consumer(s) can keep up, flow control should not be triggered.

theMayer
  • 15,456
  • 7
  • 58
  • 90
  • I disagree - I have observed an exchange fanning out to 5 separate queues. 4 queues kept up with the publish rate, but one of them got backlogged. After it reached around 4m the publish rate would intermittently halve - there were no other errors to suggest anything other than RMQ. I can only put it down to credit flow. – Rene Wooller Feb 23 '15 at 23:07
  • @ReneWooller Can you be more precise as to what you are disagreeing with in this instance? Reading your answer below, it sounds like an elaboration on what I have already outlined. Interesting, but not necessarily a disagreement. – theMayer Feb 24 '15 at 02:28
  • This statement in particular "So, if one consumer gets stuck, as long as the other consumer(s) can keep up, flow control should not be triggered." If one consumer gets stuck, and the queue it is pulling off gets backlogged, it will trigger flow control. I've just realised you could mean a cluster of consumers all pulling off the same queue - if this is what you meant, I don't disagree. – Rene Wooller Feb 24 '15 at 22:58
  • Well, that is what I was referring to- the question was unclear in that respect and I should have asked for clarification. – theMayer Feb 25 '15 at 14:23