Azure Service Bus topic: how enable concurrent readers?

Question

Background

I'm using the Azure Service Bus. Inside a namespace, I have a topic with an active subscription. I have a producer pushing messages to the topic, and I have n numbers of services subscribing to the same subscription (for scalability reasons). These services are using PeekLock() to fetch messages, performing some work on them and are then releasing the message lock through Complete() once work has finished.

Problem

Whenever a service has a message locked, no new services seem to be able to fetch messages from the same topic/subscription. It seems as if the whole topic is locked, not only the message that has a lock on itself.

The actual problem is that I have services just sitting around doing nothing but waiting for the service with an active PeekLock() to be finished.

Questions

Is there a way to enable concurrent readers (or concurrent locks) in the topic (and subscription) or queue?
The (for me unwanted) scenario which I explain above - is it a consequence of some sort of a guarantee to deliver messages in the order they are recieved?

Code snippet: Csharp

public static void main(...) 
{
  // configure the options
  OnMessageOptions messageOptions = new OnMessageOptions();
  messageOptions.AutoComplete = false;

  // look for messages
  serviceBusClient.OnMessage(msg => processMessage(msg, connectionKey), messageOptions);
}


private static void processMessage(BrokeredMessage msg, string connectionKey)
{
  try
  {
    // do stuff to the message
    //...

    // for debugging: no new clients grab messages 
    // while this client has lock active. the sleep
    // is to simulate heavy work load
    Thread.Sleep(5000);

    // release lock on message
    msg.Complete();
  }

  catch (Exception e)
  {
    msg.Abandon();
  }
}

Since this question is rather old I assume the Service Bus SDK changed because nowadays the `OnMessage()` method is exclusive to Service Bus Queues und does indeed lock the entire queue as the queue order needs to be guaranteed. Topics however don't have this guarantee and you can use `SubscriptionClient.ReceiveAsync` for example which will not lock the topic — Peter, Jan 03 '19 at 13:53

score 0 · Answer 1 · answered Apr 07 '17 at 07:53

Do you use Recieve() or Receieve(Int64) Method (or the Async variant) to receieve your message?

You have to be aware that the Locks are on the message itself, not on the Subscription/Queue. But, if you use the overload with Int64 and give so big number - you get (lock) all messages, and your other processes do not have what to do.

There is no way to lock a subscription in Azure Service Bus. The limit of concurrent subscription receievers is 5000!

Here you can read a little more about Performance in Service Bus Messaging.

Next time you ask a question, please also include a code snippet for us to be able to better understand your issue!

I use **SubscriptionClient.onMessage()**: serviceBusClient.OnMessage(msg => processMessage(msg, connectionKey), messageOptions); — H. Larsson, Apr 07 '17 at 08:12

score 0 · Answer 2 · answered Jun 29 '17 at 14:09

You could try explicitly setting the PrefetchCount on the client to 0, according to this page it is meant to default to that, but I suspect it is not if you are finding that no other clients can get messages:

public static void main(...) 
{
  // configure the options
  OnMessageOptions messageOptions = new OnMessageOptions();
  messageOptions.AutoComplete = false;

  serviceBusClient.PrefetchCount = 0;

  // look for messages
  serviceBusClient.OnMessage(msg => processMessage(msg, connectionKey), messageOptions);
}

score -1 · Answer 3 · answered Apr 07 '17 at 13:07

-1

The simplest solution in your case might be to use partitioned topics/queues, see https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning

answered Apr 07 '17 at 13:07

Slava Asipenko

392
1
3

Azure Service Bus topic: how enable concurrent readers?

Background

Problem

Questions

Code snippet: Csharp

3 Answers3