RabbitMQ asynchronous support

Question

Does the RabbitMQ .NET client have any sort of asynchronous support? I'd like to be able to connect and consume messages asynchronously, but haven't found a way to do either so far.

(For consuming messages I can use the EventingBasicConsumer, but that's not a complete solution.)

Just to give some context, this is an example of how I'm working with RabbitMQ at the moment (code taken from my blog):

var factory = new ConnectionFactory() { HostName = "localhost" };

using (var connection = factory.CreateConnection())
{
    using (var channel = connection.CreateModel())
    {
        channel.QueueDeclare("testqueue", true, false, false, null);

        var consumer = new EventingBasicConsumer(channel);
        consumer.Received += Consumer_Received;
        channel.BasicConsume("testqueue", true, consumer);

        Console.ReadLine();
    }
}

can you be a little more specific? what do you mean by "asynchronous" in this case? what are you trying to accomplish? — Derick Bailey, Aug 12 '15 at 12:21
async/await... so I'm looking for equivalents that are awaitable and return a task, like System.IO has e.g. ConnectAsync(), ReadAsync(), etc. — Gigi, Aug 12 '15 at 13:26

score 51 · Accepted Answer · edited Sep 24 '21 at 04:02

51

Rabbit supports dispatching to asynchronous message handlers using the AsyncEventingBasicConsumer class. It works similarly to EventingBasicConsumer, but allows you to register a callback which returns a Task. The callback is dispatched to and the returned Task is awaited by the RabbitMQ client.

var factory = new ConnectionFactory
{
    HostName = "localhost",
    DispatchConsumersAsync = true
};

using(var connection = cf.CreateConnection())
{
    using(var channel = conn.CreateModel())
    {
        channel.QueueDeclare("testqueue", true, false, false, null);

        var consumer = new AsyncEventingBasicConsumer(model);

        consumer.Received += async (o, a) =>
        {
            Console.WriteLine("Message Get" + a.DeliveryTag);
            await Task.Yield();
        };
    }

    Console.ReadLine();
}

edited Sep 24 '21 at 04:02

Pang

9,564
146
81
122

answered Dec 07 '17 at 11:58

Paul Turner

38,949
15
102
166

I can't quite pinpoint which release this was added in, but the relevant commits are from February 2017. – Paul Turner Dec 10 '17 at 21:26
2

Seems like it's been available since 5.0.0-pre3. – Gigi Dec 16 '17 at 14:09
@Gigi Do we have async support for Producer? – Pingpong Apr 28 '18 at 20:21
@Pingpong not that I know of. – Gigi Apr 29 '18 at 20:34
Take a look to https://gist.github.com/kjnilsson/732c0883c7807647e84ba5be2c3027f5 – Matteo1010 Mar 23 '20 at 19:07

score 13 · Answer 2 · answered Aug 12 '15 at 19:15

13

there is no async/await support built in to the RabbitMQ .NET client at this point. There is an open ticket for this on the RabbitMQ .NET Client repository

answered Aug 12 '15 at 19:15

Derick Bailey

72,004
22
206
219

2

This is really ridiculous after all these years for such a popular platform.... – Ahmad Aug 30 '21 at 06:52

score 4 · Answer 3 · edited Sep 24 '21 at 04:05

To summarize current async/TPL support:

As @paul-turner mentioned, there is now an AsyncEventingBasicConsumer which you can register events for and return a Task.
There is also an AsyncDefaultBasicConsumer for which you can override virtual methods such as HandleBasicDeliver and return a Task. Original PR here (looks like it was also introduced in 5.0?)
Per the final comments on the above PR and this issue, it looks like they are working on a new, from-scratch .NET client which would more fully support async operations, but I don't see any specific links to that effort.

score 3 · Answer 4 · edited Sep 24 '21 at 04:07

There is AsyncEventingBasicConsumer and all that it does, is awaiting your async "event handlers" when message is received. That's the only thing that is made asynchronous here. Typically you don't get any profits from this, because you have only one "handler". Messages are still processed one-by-one. They are processed synchronously! Also you lose control of exception handling because awaiting is done inside Consumer.

Let me guess that by asynchronous message processing you mean some degree of parallelism.

What I ended up using is ActionBlock from TPL Dataflow. ActionBlock runs as much tasks as you configured it to, managing awaits and parellelism. Since it operates on Tasks, not Threads, it can manage with less resources, as long as they are truly asynchronous.

Regular EventingBasicConsumer calls actionBlock.Post(something).
For parallel processing you need to tell RMQ to send you N messages before you ack them: model.BasicQos(0, N, true);
ActionBlock has options with MaxDegreeOfParallelism property which also needs to be set to N.
ActionBlock runs async Tasks which receive data posted earlier by Consumer. Tasks should not throw because ActionBlock stops all processing on exceptions.
Be careful to pass CancellationToken around and correctly wait for ActionBlock to finish all running Tasks: actionBlock.Complete(); await actionBlock.Completion;

RabbitMQ asynchronous support

4 Answers4

Linked