0

Below I have a fairly simple .NET console app consuming an Azure Service Bus Queue.

As you'll see, I am using Task.Factory to fire up 25 receiver tasks that then call my APM-style BeginMessageReceive method. Then, at the end of EndMessageReceive, I call BeginMessageReceive again to keep the loop going.

My question is how could I achieve the same sort of thing but switching from the APM-style BeginMessageReceive/EndMessageReceive to a TPL/TAP approach using recursive Tasks and possibly utilizing C# 5.0 async/await?

using System;
using System.Configuration;
using System.Threading.Tasks;
using Microsoft.ServiceBus.Messaging;


namespace ServiceBusConsumer
{
    class Program
    {
        private static QueueClient _queueClient;

        private static void Main(string[] args)
        {    
            var connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
            _queueClient = QueueClient.CreateFromConnectionString(connectionString, "MyQueue");

            for (var i = 0; i < 25; i++ )
            {
                Task.Factory.StartNew(BeginMessageReceive);
            }

            Console.WriteLine("Waiting for messages...");
            Console.ReadKey();

            _queueClient.Close();

        } //end private static void Main(string[] args)

        private static void BeginMessageReceive()
        {
            _queueClient.BeginReceive(TimeSpan.FromMinutes(5), EndMessageReceive, null);
        }

        private static void EndMessageReceive(IAsyncResult iar)
        {
            var message = _queueClient.EndReceive(iar);
            try
            {
                if (message != null)
                {
                    var msg = message.GetBody<string>();
                    Console.WriteLine("Message: " + msg);

                    if (_queueClient.Mode == ReceiveMode.PeekLock)
                    {
                        // Mark brokered message as completed at which point it's removed from the queue.
                        message.Complete();
                    }
                }
            }
            catch (Exception ex)
            {
                if (_queueClient.Mode == ReceiveMode.PeekLock)
                {
                    // unlock the message and make it available 
                    message.Abandon();
                }

                Console.WriteLine("Exception was thrown: " + ex.GetBaseException().Message);
            }
            finally
            {
                if (message != null)
                {
                    message.Dispose();
                }
            }
            BeginMessageReceive();
        }

    }
}

New modified code for recursively call itself again if the MessageReceive timeout expires:

private static async Task MessageReceiveAsync()
{
    while (true)
    {
        using (var message = await _queueClient.ReceiveAsync(TimeSpan.FromMinutes(5)))try
        {
            if (message != null)
            {               
                try
                {

                    var msg = message.GetBody<string>();
                    Console.WriteLine("Message: " + msg);

                    if (_queueClient.Mode == ReceiveMode.PeekLock)
                    {
                        // Mark brokered message as completed at which point it's removed from the queue.
                        await message.CompleteAsync();
                    }
                }
                catch (Exception ex)
                {
                    message.AbandonAsync();
                    Console.WriteLine("Exception was thrown: " + ex.GetBaseException().Message);
                }
            }
        }
    }
}
anderly
  • 727
  • 7
  • 16
  • The modified code above seems to do the trick. Still wondering if PLINQ would be better/easier for this if MessageReceiveAsync() is going to perform I/O (WebRequest) operations. Essentially, I'm trying to parallel-ize the message receive operation from the queue and then I'll need to parallel-ize the work for each message (which will effectively be an HTTP Post); – anderly Feb 07 '13 at 21:50
  • FYI, one of the reasons I have to control how many MessageReceiveAsync() operations I'm firing is because an Azure Service Bus queue has a limit on the number of concurrent connections. – anderly Feb 07 '13 at 21:53

1 Answers1

4

It looks like the Azure client libs still haven't been updated with TAP APIs. Not sure what the holdup is there...

Anyway, you can create your own APM->TAP wrappers using TaskFactory.FromAsync, as such:

public static class MyAzureExtensions
{
  public static Task<BrokeredMessage> ReceiveAsync(this QueueClient @this,
      TimeSpan serverWaitTime)
  {
    return Task<BrokeredMessage>.Factory.FromAsync(
        @this.BeginReceive, @this.EndReceive, serverWaitTime, null);
  }

  public static Task CompleteAsync(this BrokeredMessage @this)
  {
    return Task.Factory.FromAsync(@this.BeginComplete, @this.EndComplete, null);
  }
}

Once you've wrapped the Azure calls into a TAP-ready API, you can use them as such:

private static void Main(string[] args)
{    
  var connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
  _queueClient = QueueClient.CreateFromConnectionString(connectionString, "MyQueue");

  for (var i = 0; i < 25; i++ )
    MyMessageReceiveAsync();

  Console.WriteLine("Waiting for messages...");
  Console.ReadKey();

  _queueClient.Close();
}

private static async Task MyMessageReceiveAsync()
{
  while (true)
  {
    using (var message = await _queueClient.ReceiveAsync(TimeSpan.FromMinutes(5)))
    {
      try
      {
        var msg = message.GetBody<string>();
        Console.WriteLine("Message: " + msg);

        if (_queueClient.Mode == ReceiveMode.PeekLock)
        {
          // Mark brokered message as completed at which point it's removed from the queue.
          await message.CompleteAsync();
        }
      }
      catch (Exception ex)
      {
        if (_queueClient.Mode == ReceiveMode.PeekLock)
        {
          // unlock the message and make it available 
          message.Abandon();
        }

        Console.WriteLine("Exception was thrown: " + ex.GetBaseException().Message);
      }
    }
  }
}

One benefit to using async like this is that you don't tie up thread pool threads unnecessarily. The original used 25 threads to listen; my sample will not use any threads to listen. The only time a thread pool thread is tied up in my sample is when a message is being abandoned (in the error handling branch).

There is one major semantic difference from the original code: if the QueueClient's "receive" raises an exception, in the original code it would crash the process; in my example the exception would be ignored.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • Awesome! That's exactly what I wanted to achieve. Thanks a bunch Stephen! – anderly Feb 07 '13 at 14:30
  • Now, only one issue I'm running into and not sure how to solve. As you can see above, in the call to ReceiveAsync I specify a server wait timeout of 5 minutes. So, after 5 minutes of not receiving messages, my console app won't receive any further messages. I know that you can set this up to 24 days, but any idea on how to modify this to account for that and always keep 25 "listeners" going? Eventually, this will be put into an Service/Azure Worker Role, so I need it to work on a continuous loop basis. – anderly Feb 07 '13 at 19:33
  • What happens on a timeout? Does it raise an exception? – Stephen Cleary Feb 07 '13 at 20:05
  • No, the ServiceBusConsumer console app just sits there running. And then if I queue up new messages (in my other console app that queues test messages) they don't get picked up unless I restart the ServiceBusConsumer console app. Maybe it's throwing an exception in ReceiveAsync, but as you indicated, it might be getting swallowed and so I don't see it. – anderly Feb 07 '13 at 20:09
  • I've done something similar in the past with a custom ThreadPool class that has a producer/consumer queue and uses Monitor.Wait/Monitor.Pulse to avoid spinning and that's essentially what I'm trying to do using TPL/TAP. I want to always have 25 "workers" waiting (not spinning) and attempting to receive messages asynchronously from the queue. – anderly Feb 07 '13 at 20:27
  • 1
    If it throws an exception, then you can catch it. Just move the `using` inside the `try`. – Stephen Cleary Feb 07 '13 at 21:10
  • Ok, made a slight change (see new code below my original post above) that seems to account for the message wait timeout (in which case message is null) and then simply recursively calls itself again which would essentially start a new async receive method (effectively restarting the wait timeout). Let me know what you think. – anderly Feb 07 '13 at 21:32
  • Technically, it looks like with the Recursive call to MessageReceiveAsync the while(true) loop is no longer needed. Although, I'm not sure why that wasn't working since it should be looping forever trying to receive from the queue. – anderly Feb 07 '13 at 22:12
  • I usually avoid significant recursion, and definitely avoid infinite recursion. C# doesn't guarantee tail-call optimization - and they're not planning to add it, either. – Stephen Cleary Feb 08 '13 at 01:27
  • Good point, I modified the original version (see above) so that it accounts for message being null if the wait timeout expires, but avoids recursive call to MessageReceiveAsync. Seems like handling message == null was the key in either scenario. – anderly Feb 08 '13 at 13:52