17

Does RabbitMQ call the callback function for a consumer when it has some message for it, or does the consumer have to poll the RabbitMQ client?

So on the consumer side, if there is a PHP script, can RabbitMQ call it and pass the message/parameters to it. e.g. if rating is submitted on shard 1 and the aggregateRating table is on shard 2, then would RabbitMQ consumer on shard 2 trigger the script say aggRating.php and pass the parameters that were inserted in shard 1?

jeff musk
  • 1,032
  • 1
  • 10
  • 31
  • 1
    The libraries are implemented differently. Most of them have support for basic.consume. If your php library does not, you'll need to write your own. For instance, you could drive your php script by a python script or java program that consumes messages from the broker. – scvalex Feb 08 '12 at 10:39
  • The Python/Java clients wouldn't have to poll at all, but they would need a stable connection to the broker. The broker would push messages to the P/J clients. The clients could then call your scripts for each message. See the RabbitMQ tutorials for details: http://www.rabbitmq.com/getstarted.html – scvalex Feb 12 '12 at 18:51
  • 1
    I don't have any experience with PHP and I don't know anything about the PHP AMQP clients. Anything I've said about the PHP library is just an educated guess. I haven't really answered your question here. BTW, in the future, you might want to post questions like this to the RabbitMQ Discuss mailing list. I think I'm the only RabbitMQ developer that checks SO; but we make a point of answering any question posted on the mailing list. – scvalex Feb 13 '12 at 16:05

3 Answers3

12

The AMQPQueue::consume method is now a "proper" implementation of basic.consume as of version 1.0 of the PHP AMQP library (http://www.php.net/manual/en/amqpqueue.consume.php). Unfortunately, since PHP is a single threaded language, you cant do other things while waiting for a message in the same process space. If you call AMQPQueue::consume and pass it a callback, your entire application will block and wait for the next message to be sent by the broker, at which point it will call the provided callback function. If you want a non blocking method, you will have to use AMQPQueue::get (http://www.php.net/manual/en/amqpqueue.get.php), which will poll the server for a message, and return a boolean FALSE if there is no message.

I disagree with scvatex's suggestion to use a separate language for using a "push" approach to this problem though. PHP is not IO driven, and therefore using a separate language to call a PHP script when a message arrives seems like unnecessary complexity: why not just use AMQPQueue::consume and let the process block (wait for a message), and either put all the logic in the callback or make the callback run a separate PHP script.

We have done the latter at my work as a large scale job processing system so that we can segregate errors and keep the parent job processor running no matter what happens in the children. If you would like a detailed description of how we set this up and some code samples, I would be more than happy to post them.

Pieter
  • 606
  • 1
  • 5
  • 7
10

What you want is basic.consume, which allows the broker to push messages to clients.

That said, the libraries are implemented differently. Most of them have support for basic.consume, but because of inherent limitations of the frameworks used, some don't (most notably the official RabbitMQ C client on which a lot of other clients are based).

If your PHP library does not support basic.consume, you either have to use polling (bad), or you could use one of the more complete clients to drive the script. For instance, you could write a Python or Java program that consumes from the broker (so, the broker pushes deliveries to them) and they could call the script whenever a new message is received. The official tutorials are a great introduction to the AMQP APIs and are a good place to start.

This is efficient from most points of view, but it does require a stable connection to the broker.

If in doubt about the capabilities of the various clients, or if you need more guidance, the RabbitMQ Discuss mailing list is a great place to ask questions. The developers make a point of answering any query posted there.

scvalex
  • 14,931
  • 2
  • 34
  • 43
0

Pecl amqp allows to use consume functionality with AMQPQueue::consume method. You just need to pass callback function in it and it will be executed when message arrives.

wormhit
  • 3,687
  • 37
  • 46