0

When using an Azure ServiceBus Topic Subcription with multiple competing consumers, I can only get consumers to round-robin when the SubscriptionClient's PrefetchCount not set.

When specifying any PrefetchCount > 1, then under low load, the first consumer that happened to receive a message seems to be given an affinity to messages such that ALL messages are received by only that consumer.

If I add load (i.e. throw more messages concurrently at the subscription), then other consumers do get some messages, but load is not evenly distributed.

Using a Queue (and QueueClient), messages are more-or-less evenly distributed to clients -- with any PrefetchCount.

Note:

  • MaxConcurrentCalls is a a 'high' value (200)
  • I'm using PeekLock mode in all cases;
  • Subscriptions use a CorrelationFilter

This Topic-Subscription behavior doesn't seem correct to me...I was under the assumption that Queue and Subscription client competing consumers mechanism should be consistent.

What could I have configured incorrectly? Or is this expected behavior?

Ive
  • 457
  • 5
  • 19

1 Answers1

2

Both queues and subscriptions support the competing consumers in the same way. Prefetch is what not working the way you might expect. When you ask to prefetch N items, the broker will not distribute those evenly among the competing consumers. Rather will try to give each consumer as much as it can. So if you have 10 messages and set the prefetch to 10, the first consumer might get it all leaving nothing to handle to the second consumer.

Sean Feldman
  • 23,443
  • 7
  • 55
  • 80
  • I was hoping you'd have a comment, @Sean - thank you. I'd agree with you if there those N items were available _at the same time_. Then the prefetch-count (of >N) of one consumers would result in all messages being received by that consumer. But with low load -- only one message being received at a time -- why would the first consumer always take precedence? – Ive Jan 10 '20 at 12:10
  • The broker has no guarantee all the available messages to the first consumer, even with a prefetch of N and less than N messages found on the broker. It can give as much as it can at that moment. Could be one, two, or N. Unlike RabbitMQ, Service Bus doesn't try to equally distribute prefetched messages among the competing consumers AFAIK. Hence this "random" distribution. – Sean Feldman Jan 10 '20 at 16:28
  • Another suggestion would be to raise an issue with the broker team, requesting to document the behaviour. Issue tracker: https://github.com/Azure/azure-service-bus/issues – Sean Feldman Jan 10 '20 at 16:29