0

Are there any downsides to telling XREADGROUP to block until there is a message rather than the client having to poll?

From:

https://redis.io/commands/xreadgroup

It is not clear that this means:

"On the other side when XREADGROUP blocks, XADD will pay the O(N) time in order to serve the N clients blocked on the stream getting new data."

Can someone shed some light on the blocking mechanisms of streams in Redis?

Eugene Yarmash
  • 142,882
  • 41
  • 325
  • 378
mjs
  • 21,431
  • 31
  • 118
  • 200

1 Answers1

1

"On the other side when XREADGROUP blocks, XADD will pay the O(N) time in order to serve the N clients blocked on the stream getting new data."

Say, the stream is empty, and N clients call XREADGROUP with different group names. Since the stream is empty, these clients will block until there's new message.

When you call XADD to add a message to the stream, Redis need to send replies to these N blocking clients. That's why XADD will pay O(N) time.

Are there any downsides to telling XREADGROUP to block until there is a message rather than the client having to poll?

If N is very large, i.e. too many clients blocking on the stream, XADD command might block Redis for a while, since it's single-threaded. If N is small, there won't be performance impact.

for_stack
  • 21,012
  • 4
  • 35
  • 48
  • for_stack, you mean N as in consumers to notify all of them, even if the message is not added to its own stream? What will happen with those streams blocking and message added to other stream? Will they get released and get a null? – mjs Sep 02 '20 at 09:37
  • NO. Say, there're two streams: stream A and stream B, both are empty. And there're 3 clients blocking on stream A, while 5 clients blocking on stream B. If you calls XADD on stream A, all 3 clients blocking on stream A, will be unblocked. However, the 5 clients blocking on stream B, will continue blocking. So in this case, N is 3. – for_stack Sep 02 '20 at 11:59
  • Thank you for clarifying. When you say all 3 clients gets released, can I expect a null value in at least 2 of the other clients? I am using jedis, and unsure how it handles such cases. Maybe I should not expect a value from clients but check value returned first. – mjs Sep 03 '20 at 12:16
  • NO. Since these consumers have different group names, all 3 clients will be unblocked with the message you just added. – for_stack Sep 04 '20 at 00:15
  • Hmm, you mean three connections end up consuming the same message? In this case, I am assuming all three clients are the same software running on three different servers, meaning they are all running copies of the same connection setting and stream, and group. In my case, I am wondering what happens when three clients of the same stream, same group are waiting for a message and a message comes through XADD, but they are all on the same stream and group. I can understand the message being delivered to another group but that is separate so that is understandable. – mjs Sep 04 '20 at 19:18
  • 2
    In your case, each client has a unique consumer name, and the message will be consumed by one and only one consumer in the same group. Other clients continue blocking on the stream. – for_stack Sep 07 '20 at 00:38