0

I have a distributed system that reads messages from RabbitMQ. In my situation now I need to process no more than N msgs/s.

For example: Imagine service A that sends text messages (SMS). This service can only handle 100 msgs/s. My system has multiple consumers that reads data from RabbitMQ. Each message needs to be processed first and than be send to service A. Processing time is always different.

So the question:

  • Is it possible to configure queue to dispatch no more than 100 msgs/s to multiple consumers?
Vor
  • 33,215
  • 43
  • 135
  • 193

1 Answers1

2

You can use the prefetch_size parameter of the qos method to limit throughput to your consumers.

channel.basic_qos(100);

See also: http://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/

Bram Gerritsen
  • 7,178
  • 4
  • 35
  • 45
  • This is very nice article. But I have one question, Imagine situation, where my consumers are on different servers, and don't communicate with each other. Each consumer uses `basic.get()` (that returns only one msg from queue). Will `prefetch_size` work with `basic.get()`? – Vor Jul 14 '13 at 14:34
  • I have never used `basic.get`, but I think it will not work, because you are just picking one message from the queue at a time with basic.get and poll for the next one. Why don't you use `basic.consume` in your workers? `consume` will also be more performant and better scalable from what I read. – Bram Gerritsen Jul 14 '13 at 14:51