18

In a web application, if I need to write an event to a queue, I would make a connection to redis to write the event.

Now if I want another backend process (say a daemon or cron job) to process the or react the the publishing of the event in redis, do I need a persistant connection?

Little confused on how this pub/sub process works in a web application.

Alan Plum
  • 10,814
  • 4
  • 40
  • 57
codecompleting
  • 9,251
  • 13
  • 61
  • 102
  • Some simple (1 line patch) ideas to add persistence for PubSub natively in Redis - http://abhinavsingh.com/customizing-redis-pubsub-for-message-persistence-part-2/ – Abhinav Singh Apr 25 '15 at 21:32

2 Answers2

47

Basically in Redis there are two different messaging models:

  • Fire and Forget / One to Many: Pub/Sub. At the time a message is PUBLISH-ed all the subscribers will receive it, but this message is then lost forever. If a client was not subscribed there is no way it can get it back.
  • Persisting Queues / One to One: Lists, possibly used with blocking commands such as BLPOP. With lists you have a producer pushing into a list, and one or many consumers waiting for elements, but one message will reach only one of the waiting clients. With lists you have persistence, and messages will wait for a client to pop them instead of disappearing. So even if no one is listening there is a backlog (as big as your available memory, or you can limit the backlog using LTRIM).

I hope this is clear. I suggest you studying the following commands to understand more about Redis and messaging semantics:

  • LPUSH/RPUSH, RPOP/LPOP, BRPOP/BLPOP
  • PUBLISH, SUBSCRIBE, PSUBSCRIBE

Doc for this commands is available at redis.io

antirez
  • 18,314
  • 5
  • 50
  • 44
  • so I need a persistant connection as a subscriber? – codecompleting Oct 06 '11 at 14:33
  • 5
    With Pub/Sub, yes. Look at [this](http://blog.joshsoftware.com/2011/01/03/do-you-need-a-push-notification-manager-redis-pubsub-to-the-rescue/) for an example on how to achieve persistent messages with Pub/Sub and a custom ruby client. – cbrauchli Oct 08 '11 at 15:24
2

I'm not totally sure, but I believe that yes, pub/sub requires a persistent connection.

For an alternative I would take a peek at resque and how it handles that. Instead of using pub/sub it simply adds an item to a list in redis, and then whatever daemon or cron job you have can use the lpop command to get the first one.

Sorry for only giving a pseudo answer and then a plug.

Tim Barrass
  • 4,813
  • 2
  • 29
  • 55
rm-rf
  • 1,313
  • 1
  • 15
  • 24