0

I am working on an IoT application and using C# events to subscribe to MQTT messages. I am using a library that raises an event when a message is received. I have written an Asynchronous EventHandler for processing the message.

The rate at which the messages are being received is higher than the rate of processing them. In this case, I see a delay in the processing of messages, even though the EventHandler is asynchronous.

My question is does C# internally maintain some queue for EventHandlers? If, so can I access that queue for diagnostics purposes? By accessing that queue, I can analyze the exact load which causes the delay.

EDIT

Following is my setup,

  1. I have .NET Framework 4.6.2 console application.
  2. In the application I use MQTTnet library for subscribing to MQTT messages.
  3. Following is the code I use for subscribing,
var mqttFactory = new MqttFactory();

var client = mqttFactory.CreateMqttClient();

client.ApplicationMessageReceivedAsync += async e =>
{
    // Process the message. Ex: Update database
}

var clientOptions = mqttFactory.CreateClientOptionsBuilder()
    .WithTcpServer("mqtt-broker-host", 9000)
    .Build();

await client.ConnectAsync(clientOptions);
await client.SubscribeAsync("mqtt-topic");
  1. In the handler I process the message. The main part of processing is updating the records in the database.
Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
Hem Bhagat
  • 323
  • 3
  • 12
  • 2
    Could you illustrate your question with some sample code for your current setup? – ProgrammingLlama Jun 27 '23 at 09:48
  • 2
    If the events are being handled in a UI context they are likely being processed via the Windows message queue. But more information is needed. – Matthew Watson Jun 27 '23 at 09:49
  • As requested, I have added the information regarding my setup. – Hem Bhagat Jun 27 '23 at 10:08
  • 1
    How does MQTTnet actually work? I assume the queuing is happening in there. Does it process the messages sequentially? I assume it invoked the delegate and waits for completion before sending the next message? If so, then is there an option to change that so it processes messages in parallel? – DavidG Jun 27 '23 at 10:19
  • 3
    Is this of any relevance? https://github.com/dotnet/MQTTnet/discussions/1589 – Matthew Watson Jun 27 '23 at 10:23
  • Thanks, @MatthewWatson. This seems useful. I will try the solutions listed there. – Hem Bhagat Jun 27 '23 at 10:31
  • @DavidG, MQTTnet does not wait for messages to complete. It just notifies the program when a message is received. It is "fire and forget" from the MQTTnet side. – Hem Bhagat Jun 27 '23 at 10:33
  • I am skeptical that you'll solve your problem by parallelizing the database operations. If your database can't keep up with the frequency of the MQTTnet events one-by-one, most likely it won't be able to keep up either if you bombard it with multiple events concurrently. – Theodor Zoulias Jun 27 '23 at 12:54

2 Answers2

3

EventHandler is intended to be synchronous; there is no queue of any kind. If you want some kind of queuing mechanism, maybe consider Channel<T>, which is an async queue-like device intended for independent write ("pub") and read ("sub"); or if your requirements are more complex, a range of in-process and out-of-process message-queue / event broker tools are available.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • Can you please explain how C# internally makes calls to EventHandlers when the same event is raised multiple times in a short interval? Let's say I raise 5 events in one second and the rate at which the handler can process the event is 1 second. What will be the behavior in this case? Also, does the behavior differ if I register an async handler? – Hem Bhagat Jun 27 '23 at 10:15
  • @HemBhagat a delegate is invoked synchronously, period; if you're invoking it 5 times in a second, and it takes 1s each, then I can infer that you have 5 threads in play; each of those threads will execute the delegate on it's own thread, overlapped and concurrently; there is no queue or attempt at a queue: it *just runs*; I'd need to see *exactly* what you mean to discuss "async handler", because: adding `async` doesn't (by itself) make anything async - all `async` does is put the compiler into state-machine mode where a method can consume awaitables, and itself be awaitable (by return type) – Marc Gravell Jun 27 '23 at 13:25
  • 1
    By "async handler" I mean following, The event signature provided by the MQTTnet library is, `event Func ApplicationMessageReceivedAsync;` And I am registering a handler as follows, `client.ApplicationMessageReceivedAsync += async e => {}` – Hem Bhagat Jun 27 '23 at 13:38
  • You mentioned, 5 threads will be in play, concurrently. Is there a limit to the number of concurrent threads? I see a delay when I get too many messages. To measure delay I compare the timestamp of the message sent with the current time when the handler starts processing the message and logged the difference. In the logs, I can see a delay. This means that when there are too many messages, the handler delegate is called after some delay. If I can get a new concurrent thread for each message, this problem should not occur. – Hem Bhagat Jun 27 '23 at 13:44
  • 1
    any limit there is in your code, or is simply CPU starvation; there is nothing that even tracks threads passing through delegates - it *just runs*. Invoking a delegate doesn't cause a thread to be created - it runs on the thread doing the invoking. Of course, if *your* code is spawning lots of threads, that will have problems in itself. – Marc Gravell Jun 28 '23 at 09:05
  • If you invoke a delegate that has a Task return type, then two things apply: if multiple subscribers are hooked, the behaviour is basically undefined, but: with a single consumer as long as you await the invoke, each invoke should work, but will work entirely independently of (and concurrently with) any parallel invokes from other threads. But: because of the multi-subscriber problem, events are not good with async – Marc Gravell Jun 29 '23 at 07:49
1

My question is does C# internally maintain some queue for EventHandlers?

No, an event handler is essentially just a list of delegates. And raising an event just loops over all the delegates and calls them in turn.

Adding async to a method does not automatically make anything asynchronously. It just allow you to await asynchronous methods, but sometimes methods lie about being asynchronous.

There are some queues in C#, like the message queue for the UI thread, but I do not think this is relevant in this particular.

If, so can I access that queue for diagnostics purposes. By accessing that queue, I can analyze the exact load which causes the delay.

This sounds like you want to use a performance profiler. This should tell you how much CPU time is spent in each portion of the code, and if some code is blocking for some reason. The best solution would be if you could solve your performance problem by optimization or avoid doing anything "slow".

JonasH
  • 28,608
  • 2
  • 10
  • 23