18

With RabbitMQ, is there a way to "push" messages from a queue TO a consumer as opposed to having a consumer "poll and pull" messages FROM a queue?

This has been the cause of some debate on a current project i'm on. The argument from one side is that having consumers (i.e. a windows service) "poll" a queue until a new message arrives is somewhat inefficient and less desirable than the idea having the message "pushed" automatically from the queue to the subscriber(s)/consumer(s).

I can only seem to find information supporting the idea of consumers "polling and pulling" messages off of a queue (e.g. using a windows service to poll the queue for new messages). There isn't much information on the idea of "pushing" messages to a consumer/subscriber...

user1431072
  • 1,272
  • 2
  • 13
  • 32
  • You could certainly have an out-of-band mechanism to notify consumers to start looking for stuff in the queue, but why even have a queue if your tasks are so infrequent that you're worried about the efficiency of _idling_? – U2EF1 Oct 23 '14 at 05:39
  • @U2EF1 But even then, the out-of-band mechanism to notify consumers has to use some type of "polling" to watch for new messages, doesn't it? – user1431072 Oct 23 '14 at 05:42
  • Yes, probably `select` or `kqueue` at the lowest level. So, you guys should probably not worry about it and build your thing. I'm 95% sure that polling rabbitmq is not going to be the bottleneck in your system. – U2EF1 Oct 23 '14 at 05:53
  • 1
    In AMQP `basic.consume` works exactly as you asked - consumer subscribes to receive messages from specific queue and then broker notifies consumer with messages from that queue, from head to tail (FIFO). And why having consumers are inefficient? – pinepain Oct 23 '14 at 06:54

4 Answers4

11

Having the server push messages to the client is one of the two ways to get messages to the client, and the preferred way for most applications. This is known as consuming messages via a subscription.

The client is connected. (The AMQP/RabbitMQ/most messaging systems model is that the client is always connected - except for network interruptions, of course.)

You use the client API to arrange that your channel consume messages by supplying a callback method. Then whenever a message is available the server sends it to the client over the channel and the client application gets it via an asynchronous callback (typically one thread per channel). You can set the "prefetch count" on the channel which controls the amount of pipelining your client can do over that channel. (For further parallelism an application can have multiple channels running over one connection, which is a common design that serves various purposes.)

The alternative is for the client to poll for messages one at a time, over the channel, via a get method.

davidbak
  • 5,775
  • 3
  • 34
  • 50
3

You "push" messages from Producer to Exchange.
https://www.rabbitmq.com/tutorials/tutorial-three-python.html

BTW this is fitting very well IoT scenarios. Devices produce messages and sends them to an exchange. Queue is handling persistence, FIFO and other features, as well as delivery of messages to subscribers.

And, by the way, you never "Poll" the queue. Instead, you always subscribe to publisher. Similar to observer pattern. Generally, I would say genius principle. So it is similar to post box or post office, except it sends you a notification when message is available.

Pang
  • 9,564
  • 146
  • 81
  • 122
OSP
  • 1,458
  • 1
  • 14
  • 15
  • This makes the most sense to me. "You get a notification pushed that a message is available" not the message itself pushed. Cheers. – james-see Oct 08 '19 at 01:09
  • 1
    No, you actually get the message. I think the answer meant to say you don't poll for it - the RabbitMQ framework gives you a callback (that's the "notification") but the callback has the message. – davidbak Nov 25 '20 at 16:11
  • Shouldn't that depend on the use-case? Whether the entire message needs to be pulled after it's become available. – Swapnil Dec 15 '21 at 11:47
1

Quoting from the docs here:

AMQP brokers either deliver messages to consumers subscribed to queues, or consumers fetch/pull messages from queues on demand.

And from here:

Storing messages in queues is useless unless applications can consume them. In the AMQP 0-9-1 Model, there are two ways for applications to do this:

Have messages delivered to them ("push API")
Fetch messages as needed ("pull API")

With the "push API", applications have to indicate interest in consuming messages from a particular queue. When they do so, we say that they register a consumer or, simply put, subscribe to a queue. It is possible to have more than one consumer per queue or to register an exclusive consumer (excludes all other consumers from the queue while it is consuming).

Each consumer (subscription) has an identifier called a consumer tag. It can be used to unsubscribe from messages. Consumer tags are just strings.

Vikas Prasad
  • 3,163
  • 5
  • 28
  • 39
-2

RabbitMQ broker is like server that wont send data to consumer without consumer client getting registering itself to server. but then question comes like below

Can RabbitMQ keep client consumer details and connect to client when packet comes?

Answer is no. so what is alternative well then write plugin by yourself that maintain client information in some kind of config. Plugin will pull from RabbitMQ Queue and push to client.

Please give look at this plugin might help.
https://www.rabbitmq.com/shovel.html

Frankly speaking Client need to implement AMQP protocol to receive so and should listen connection on some port for that. This sound like another server.

Regards,
Vishal

Vishal Bhandare
  • 105
  • 2
  • 5
  • I'm not completely convinced you understand the proper use cases for RabbitMQ – Shayne Oct 31 '17 at 02:42
  • I don't understand the negative votes: this is exactly what I was looking for, a way to have peripheral rabbitmqs forwarding messages to a central collector, be it another rabbitmq or my own destination service. – think01 Jan 21 '21 at 18:11