Azure Service Bus Queue Sending Freezing

Question

We were load testing an Azure queue using multiple processes. We noticed that all of the machines would simultaneously pause while sending messages to the queue.

The pauses are of different length. Most frequent are half second pauses, but we've seen pauses on all senders up to a hand-full of seconds.

Since I originally posted, we've seen the same behavior with a Java client using the HTTP bindings.

Additionally, we've seen pauses which affect every instance except one. While several of the executing programs will stop simultaneously, one will continue to send at the same rate.

In all cases, after the pause is over, we see all of the instances resume at the old rate. We start all the instances by hand over a period of several seconds.

The .NET code looks like:

using (var memoryStream = new MemoryStream())
{
    using (var streamWriter = new StreamWriter(memoryStream))
    {
        streamWriter.Write(messageText);
        streamWriter.Flush();
        memoryStream.Position = 0;

        var message = new BrokeredMessage(memoryStream, false);
        message.Properties["Name"] = "DeviceStatusProbed";
        message.Properties["MessageId"] = messageIdText;

        sender.SendAsync(message);

        if (messageNumber == 0)
        {
            stopwatch.Start();
        }

        messageNumber++;

        Console.WriteLine("MPS: " + 1000 * messageNumber / (double)stopwatch.ElapsedMilliseconds);

        Console.WriteLine("Sent message with MessageID = " + message.MessageId);
    }
}

What are the potential causes in the Azure environment for such freezes? If the cause isn't expected, is there tooling or guidance available to help us determine the cause?

Thanks.

Any code or example to share? – iCollect.it Ltd Jan 31 '14 at 09:23 — iCollect.it Ltd, Jan 31 '14 at 09:23
What data type is the messageText object? – MikeWo Feb 01 '14 at 15:19 — MikeWo, Feb 01 '14 at 15:19
String messageText; // ~250 characters in length – No One Feb 03 '14 at 14:16 — No One, Feb 03 '14 at 14:16

score 0 · Answer 1 · answered Feb 10 '14 at 06:15

0

can you provide more information about how long is the "pause".. do you see a failure in the Send call ?? How long did you see the pause ?

There isn't any reason why you should see such pauses .

It is possible that the call times out after 60 seconds (default) or just fails if there is an upgrade in SERVICEBUS or any Azure infrastructure. But upon retry it should succeed.

answered Feb 10 '14 at 06:15

Kartik Paramasivam

1

I've added that additional detail to the question. Let me know if there is any other information that could help. None of the calls are producing errors. The queue isn't full and we aren't reaching the queue connection limits. The behavior is consistent across runs, languages, and machines. – No One Feb 10 '14 at 13:28

score 0 · Answer 2 · answered Jun 15 '14 at 12:14

It might be an issue with the stream as you are wrapping the MemoryStream in a using statement.

Since the SendAsync is called asynchronously the memory stream might have already been disposed before being sent on the service bus.

Can you try and remove the using statement on the memory stream and instead call:

var message = new BrokeredMessage(memoryStream, ownsStream: true);

Azure Service Bus Queue Sending Freezing

2 Answers2