Message Queue in PHP

Question

I have developed a rest api which accepts some data and store it in a message queue (Redis List). Now from redis this data is pushed into MySQL database. The problem is client needs to wait till the data is written into mysql.

I want that the client should wait till the data is written into message queue (Redis List) and the function to push the data into MySQL should execute Asynchronously. How can I do it ? My entire code base is in PHP, So I would prefer it in PHP.

I have read this but havn't tried.

Distributed queue example in PHP using Redis

I have confusion that how slave.php ( mentioned in the link) will be executed. I mean When a new message arrives in the queue, how slave.php will find this.

I dont want to use cronjob for this. Instead when a new message arrives slave.php should get executed asynchronously. How to do it?

Manu Manjunath · Accepted Answer · 2015-02-24T16:18:19.583

14

Yes, this is very much possible using Memqueue, Redis etc.

Using Redis, this is how one can do it:

This PHP file pushes the message into queue when it gets it:

/*
    Code logic
*/
$redis->lPush("message_queue", "message 1");

A slave.php which does a "blocking pop" using brPop:

$redis = new Redis();
$redis->pconnect();

while (true) {
    list($queue, $message) = $redis->brPop(["message_queue"], 0);
    /*
    Logic to insert $message to MySQL
    */
}

Now whenever a new message arrives, the slave.php will catch it and push it to MySQL.

Don't be confused by the while(true) - above code is not running in an infinite loop. The brPop asynchronously waits till there is a new message in the queue.

I'm using PHP 5.4 and connecting to Redis 2.6 and above works fine. Better, you can have multiple slave.php files and the load gets distributed.

For more details: http://redis.io/commands/blpop

edited Feb 24 '15 at 16:18

answered Dec 24 '13 at 07:29

Manu Manjunath

6,201
3
32
31

3

Hii Manu..Thanks for your answer. It is working for me. But there is one problem. When the list is empty, slave.php waits only for few minutes after that it throws some exception. And once blocking the connection it doesn't listen even when new data arrive in the list. I have added a new question where my codes and error messages are mentioned. here is the link : http://stackoverflow.com/questions/20972743/error-in-implementing-message-queue-using-redis-error-in-using-blpop Can you please help? Thanks – Neel Kamal Jan 07 '14 at 13:51
@NeelKamal or anyone else looking for timeout other question, I have provided answer over there https://stackoverflow.com/questions/20972743/error-in-implementing-message-queue-using-redis-error-in-using-blpop/20988340#comment107877993_20988340 – Airy Apr 02 '20 at 15:46

score 0 · Answer 2 · answered Dec 23 '13 at 10:38

0

Your question is far from clear on how your system currently works.

when a new message arrives slave.php should get executed asynchronously.

This statement is an oxymoron. If processing of a message occurs asynchrhnously then it is by definition not tied to the creation of the mesage.

I suspect that the problem is that you've got no daemon handling the dequeueing of messages - but explaining how to write such a program is way beyond the scope of an answer here.

answered Dec 23 '13 at 10:38

symcbean

47,736
6
59
94

that statement is reagrding my doubt after reading this link http://redis4you.com/code.php?id=012. The system workflow is mentioned in first and second paragraph – Neel Kamal Dec 23 '13 at 10:46

Message Queue in PHP

2 Answers2