1

What is the message count limit that can be published to service bus topic at a time using TransactionScopeOption as Suppress.i am publishing message in batches
I tried publishing 300 messages in a batch..they got published
but- with 400 message count they are not getting published
the size of both the batches are less than 256 kb.
using .net core 3.1
is there a limit on number of messages along with size as well?

    var topicClient= new TopicClient(this.servicebusConnectionString, name);    
    using (var scope = new TransactionScope(TransactionScopeOption.Suppress, TransactionScopeAsyncFlowOption.Enabled))
       {
                            
            foreach (var batch in GetBatches(messageList, BatchSize))
            {
                await this.topicClient.SendAsync(batch.ToList()).ConfigureAwait(false);
            }
             scope.Complete();
       }

for creating batches I am using below function:

        public static IEnumerable<IEnumerable<Message>> GetBatches(IList<Message> source, long bucketSize)
    {
        List<Message> bucket = null;
        long size = 0;
        foreach (var item in source)
        {
            if (bucket == null)
            {
                bucket = new List<Message>();
            }

            size += item.Size;
            if (size <= bucketSize * ConvertToBytes)
            {
                bucket.Add(item);
                continue;
            }

            yield return bucket;
            bucket = new List<Message> { item };
            size = item.Size;
        }

        if (bucket?.Count > 0 && size <= bucketSize * ConvertToBytes)
        {
            yield return bucket;
        }
    }
n1k1
  • 51
  • 6
  • What .NET SDK are you using? If the latest, how is your `ServiceBusClient` configured? A tiny code snippet is not going to provide all that information. And what do you mean by "and the size of both the batches"? Mind to review and update your question to allow those that want to help have a better understanding of what are you facing? Thanks. – Sean Feldman Dec 10 '21 at 22:32
  • what exception do you get when attempting to send 400 ? batch limit is 4500 messages per batch, but there is a limit of 100 messages per transaction https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-quotas – Jdresc Dec 13 '21 at 05:56
  • @SeanFeldman I have added the required info...What I am trying to do is basically create batches(from a list of messages) of less than 256 kb and publish them. I am using Microsoft.Azure.ServiceBus package. – n1k1 Dec 13 '21 at 14:04
  • yes @DeepDave-MT..better to use Azure.Messaging.ServiceBus – n1k1 Dec 20 '21 at 05:23

1 Answers1

1

When dispatching messages, it matters if those are dispatched using Send-Via feature (lately renamed as a cross-entity transaction) or not. When the cross-entity transaction is enabled, the maximum number of messages cannot exceed 100 or the maximum single message size, 256KB when using the standard tier. The Premium tier supports up to 1MB.

When dispatching w/o cross-entity transaction, the total size is the only thing that matters. But a message size is not just its Size property. That's not enough as the entire message with all the user headers and message system properties contribute to the overall size of the payload sent to the broker.

Unfortunately, with the older versions of the Azure Service Bus SDKs, such as Microsoft.Azure.ServiceBus there's no way to get an accurate size of a message to build up a batch for safe sending. You can somewhat estimate and pad the size if you'd like but it's very "lossy".

The latest SDK, Azure.Messaging.ServiceBus has support for a batched sending and it does it right by introducing a concept of ServiceBusMessageBatch and allowing adding messages as long as their serialized size is not exceeding the maximum. Using this SDK will help you to populate your batches to the maximum and send those w/o running into a risk of a failed send operation due to the size of the payload exceeding the maximum.

Sean Feldman
  • 23,443
  • 7
  • 55
  • 80