Masstransit use RabbitMQ is very slow performance?

Question

I used RabbitMQ without Masstransit and send 10,000 message/per sec and One million message in 100 second.

But after using Masstransit with RabbitMQ the performance is very low in my machine.

The hard disk is very active (99% usage) when publish/consume message and CPU activity for this process is almost 0%.

When the run Publisher/Subscriber console application with this code :

var bus = ServiceBusFactory.New(x =>
{
    x.UseRabbitMq();
    x.ReceiveFrom("rabbitmq://localhost/Example_Hello");
});
var message = new MyMessage() { Text = "hello", When = DateTime.Now };
for (int i = 0; i < 100; i++)
{
    bus.Publish<MyMessage>(message, x => { });
}

Published 100 message in 6 second and I don't know why is very slow.

My machine's configuration and software version is:

Windows 8.1 64bit

Intel Core i3 3.30GHz

Memory 8GB

Visual Studio 2013 C#.Net 4.5.1

Erlang 6.3

RabbitMQ 3.4.4

Masstransit 2.9.9

RabbitMQ.Client 3.4.0

FYI, the ability to not wait for an Ack with .NET 4.x was added in MT 2.10. — Chris Patterson, May 21 '15 at 03:45
@MohammadRadmanFar: have you by chance succeeded here? I've just posted on mt discuss group (https://groups.google.com/forum/#!topic/masstransit-discuss/XiqSDnJzd8U) as I can't find a way around this issue. — Wiktor Zychla, Mar 17 '16 at 12:29
@ChrisPatterson: Chriss, is there a chance you could take a look at this and comment? Thanks in advance. — Wiktor Zychla, Mar 18 '16 at 07:49
I've seen speeds much higher than you're seeing. I suspect there is something else slowing you down. Do you have a repro? Can you run the MassTransit-Stress app to test your broker throughput? — Chris Patterson, Mar 19 '16 at 22:41
@ChrisPatterson: I've updated my discussion group post with a follow up containing my stress tool results. Still need a hand here, though. — Wiktor Zychla, Mar 21 '16 at 10:18
Were all your messages consumed before you ran the test with masstransit? — cantSleepNow, Mar 22 '16 at 14:49
@cantSleepNow: found the culprit, it's in the sources of MT2. See my answer. — Wiktor Zychla, Mar 23 '16 at 12:23

score 10 · Answer 1 · answered May 04 '15 at 13:59

10

This is because under the covers, MassTransit is waiting for RabbitMQ to Ack the message, ensuring that it was successfully accepted by the broker before returning to the caller. Without this wait, if the broker fails to receive the write, the message could be lost.

With MassTransit v3 (currently in pre-release), the Publish method (and all Send methods) are now async, returning a Task that can be awaited (or not, if you don't care about the outcome).

I could add a PublishAsync method for .NET 4.0 developers to version 2.9.9, in fact, that's what I may end up doing as a temporary workaround for applications still using the current stable version. It may also be useful to add a NoWait option to the SendContext, allowing the application to opt-out of the Ack wait behavior.

I just favor durability over performance in my use case, honestly.

answered May 04 '15 at 13:59

Chris Patterson

28,659
3
47
59

3

Since I pretty much answered to myself (and it turned out a simple bug was in sources) I could as well not award the bounty to anyone. However, since the bounty was already set up, I decided to award it to you Chris as a token of gratitutde for all your effort you put in MT. Keep up the good work, regards :) – Wiktor Zychla Mar 28 '16 at 15:58
It seems for me, masstransit creates connection every publish. If using something like RabbitMq.Client, connection is get created explicitely, so when publish cycle is inside connection - speed is about 5 times faster, than it's above. When ACK is turned on it's about the same speed like masstransit (about 50 messages/sec) on my PC – Lapenkov Vladimir Sep 17 '21 at 13:49
The bus should be started prior to publishing messages, and the message publish rate can easily reach 15,000+ messages using MassTransit. If you’re publishing in a loop, and awaiting each call to Publish, well, that’s your mistake. Don’t do that, async 101. – Chris Patterson Sep 18 '21 at 03:38

score 5 · Answer 2 · answered Mar 23 '16 at 12:22

I have found a bug in MT2 <= 2.10.1 that prevents it from correctly handling the WaitForAck flag. I posted a patch proposal and hope Chris releases 2.10.2 as soon as possible.

The detailed information on the issue is described here:

https://groups.google.com/forum/#!topic/masstransit-discuss/XiqSDnJzd8U

In short, the issue is caused by the bug in the SendContext copy constructor, despite the original context has the wait for ack flag set to false, the context that is used in the call has the flag always set to true.

Masstransit use RabbitMQ is very slow performance?

2 Answers2

Linked