When do Redis PUB/SUB events fire-and-forget?

Question

Redis events are fire-and-forget. If no one is listening, they will just remain undetected. There's no history. But nowhere on the internet did I find anything that answers the following question (I'm using redis-py and aioredis for python):

Does the code actually need to LISTEN to the events to receive it or will the redis connection cache events for me?

Let's suppose I have this loop:

redis_db = Redis(...)

while True:
    print("Hey, I'm listening now!")
    msg = redis_db.blocking_subscribe("some_topic") # pseudo-code, because I'm using various libraries
    if msg == "calc":
        do_a_heavy_5_seconds_calculation()
        print("YAY, I was busy and am back up now!")

So will I miss all events that occur in the 5s calculation or will my redis_db cache them for me?

In other words... does fire-and-forget apply if I don't have a connection or if I have a connection, but am not listening ?

score 2 · Answer 1 · answered Feb 20 '20 at 13:31

As you mentioned, if you are using Redis Pub/Sub, it is a fire and forget so your application has to be up and running and listen to the event.

I have to say that I have not tested your use case.

For your logic, to have more control, I would suggest you look at Redis Streams.

First of all, this is not a fire-and-forget publish-subscribe, ut a more resilient messaging system.
Your application will control which messages to read, have to be read and send an ack when read.

Your pseudo-code will be very easy to implement with Streams, with better control.

Hello, thanks for your efforts to write this answer. Well, my basic example was just for explanation. In reality, Would you be so kind and also answer a more specific question I asked a while ago? That would really help me getting started with Redis Streams: https://stackoverflow.com/questions/60248078 . Thanks very much. +1 — fameman, Feb 20 '20 at 18:37

smassey · Answer 2 · 2020-02-20T19:42:17.057

1

It's a bit of both. You need a connection AND that connection must be in a "subscription" state. As soon as execute the "blocking_subscribe" method, you have effectively changed that connections state to the "subscription state" so redis will continue to buffer these messages for you regardless if your python thread is busy processing other messages at the same time. Now of course, if it crashes for any reason, then that connections server side buffer will be lost.

edited Feb 20 '20 at 19:42

answered Feb 20 '20 at 13:41

smassey

5,875
24
37

Hey there. Thanks for your clear answer. Did you actually try it out, is it somewhere in the docs where I haven't seen it, or is it more of an assumption? – fameman Feb 20 '20 at 18:38
That's just how redis works. For details, it's explained here: https://redis.io/topics/clients – smassey Feb 20 '20 at 19:51

score 0 · Answer 3 · answered Feb 20 '20 at 13:37

"Fire and forget" applies to the server's contract with the client. It means that the message will be sent only once, assuming that the client is connected. If the message is lost in the network and never gets to the client, then that's that. If the message gets to the client and the client process/server crashes before reading it, then that's that.

In your case, the client (redis-py) listens to the network socket that makes the connection to the server. The socket itself is handled by the OS and buffers the incoming messages. As long as you keep draining the socket by reading from it messages aren't lost.

When do Redis PUB/SUB events fire-and-forget?

3 Answers3