3

I have simplified the scenario as that I have as good as possible. Down below you find the complete code with which you can test the program yourself. In my use case I have to wait for a reset event from another Thread - lets call that subEvent. The event subEvent is signaled once again another thread is finished. Lets call this resetEvent. In this simplified version I signal resetEvent after 200ms and that signals subEvent immediately. This is implemented in the SingleCascaded method.

This example works as expected. The output for me is

Sub Thread: 229 ms
Main 229 ms
Thread: 229 ms

Well the 30ms is still a heavy price, but it becomes far worse, when I increase the number of waits of both reset events - this is closer to the scenario I have in my program. In the method MultipleCascaded I create 5 threads that wait for the resetEvent - which is set after 200 ms - and 15 threads that wait for the subEvent - which is set after the resetEvent.

Set reset event 641
Main 641 ms
Thread 4: 660 ms
Thread 1: 661 ms
Thread 3: 662 ms
Sub Thread 4.0.: 662 ms
Sub Thread 4.1.: 663 ms
Sub Thread 0.0.: 661 ms
Sub Thread 0.1.: 661 ms
Sub Thread 1.1.: 664 ms
Sub Thread 1.2.: 664 ms
Sub Thread 3.0.: 665 ms
Sub Thread 3.1.: 665 ms
Sub Thread 3.2.: 666 ms
Thread 0: 661 ms
Sub Thread 4.2.: 663 ms

The interesting part here is, that the Set\Wait method of the reset event does not produce the time gap. It seems that in this case the timer cannot execute after the designated time.

Of course in my application these are no timers, things like I/O access, computation and the like. But the effect is the same. There is a visible time gap in the execution where seemingly nothing happens. I had the issue in other scenarios as well (overly use of async/await for example). But there I could not reproduce it in a small example.

My question now is. What is happening? And far more important, how can I work around that? Here is what I have found out what is not the problem:

  • All threads in the thread pool are blocked - There are a lot of threads that can be active at the same time
  • The timer - as I mentioned in my actual application I do I/O accesses and computational work. There are no timers.
  • The implementation of this particular reset event - This happens also when I use ManualResetEvent and AutoResetEvent and in my old application I worked with async/await; in this case it is a TaskCompletionSource

Here now the code for you to test it.

class Program
{
    static void Main(string[] args)
    {
        SingleCascaded();
        MultipleCascaded();
    }

    private static void MultipleCascaded()
    {
        Console.WriteLine("Test multiple waits cascaded");
        ManualResetEventSlim resetEvent = new ManualResetEventSlim(false);
        Stopwatch watch = new Stopwatch();
        watch.Start();
        for (int j = 0; j < 5; j++)
        {
            ThreadPool.QueueUserWorkItem(state =>
            {
                ManualResetEventSlim subEvent = new ManualResetEventSlim(false);
                for (int k = 0; k < 3; k++)
                {
                    ThreadPool.QueueUserWorkItem(state =>
                    {
                        subEvent.Wait();
                        Console.WriteLine(
                            $"Sub Thread {((Tuple<int, int>) state).Item1}.{((Tuple<int, int>) state).Item2}.: {watch.ElapsedMilliseconds} ms");
                    }, new Tuple<int, int>((int) state, k));
                }

                resetEvent.Wait();
                subEvent.Set();
                Console.WriteLine($"Thread {state}: {watch.ElapsedMilliseconds} ms");
            }, j);
        }

        using Timer timer = new Timer(state =>
                                      {
                                          Console.WriteLine($"Set reset event {watch.ElapsedMilliseconds}");
                                          resetEvent.Set();
                                      }, null, 200,
                                      Timeout.Infinite);
    }

    private static void SingleCascaded()
    {
        Console.WriteLine("Test single waits cascaded");
        
        ManualResetEventSlim resetEvent = new ManualResetEventSlim(false);
        Stopwatch watch = new Stopwatch();
        watch.Start();
        
        ThreadPool.QueueUserWorkItem(state =>
        {
            ManualResetEventSlim subEvent = new ManualResetEventSlim(false);
            ThreadPool.QueueUserWorkItem(state =>
            {
                subEvent.Wait();
                Console.WriteLine(
                    $"Sub Thread: {watch.ElapsedMilliseconds} ms");
            });

            resetEvent.Wait();
            subEvent.Set();
            Console.WriteLine($"Thread: {watch.ElapsedMilliseconds} ms");
        });

        using Timer timer = new Timer(state => { resetEvent.Set(); }, null, 200,
                                      Timeout.Infinite);
        resetEvent.Wait();
        Console.WriteLine($"Main {watch.ElapsedMilliseconds} ms");
    }
}
Yggdrasil
  • 1,377
  • 2
  • 13
  • 27
  • It seems the cascaded call is not necessary. It works too if I have just one ResetEvent and many threads waiting for it. – Yggdrasil Jul 13 '21 at 10:23
  • I think this is both related to the way that `ManualResetEventSlim` waits ("busy spinning for a short time"), as well as the fact that the Timer callback runs on ThreadPool and gets queued for execution when a thread becomes available, which is not immediately, but in your case 600ms and not 200ms as expected. Even if you do not have a Timer in your app, if you're running your I/O on ThreadPool, it's the same problem. Not sure if this is still relevant: https://learn.microsoft.com/en-us/archive/msdn-magazine/2010/september/concurrency-throttling-concurrency-in-the-clr-4-0-threadpool – evilmandarine Jul 18 '21 at 16:39

1 Answers1

1

As per my comment, this is due to the fact that the loop uses all available threads in the ThreadPool which then needs to start adding more threads using an algorithm, and that the Timer callback (resetevent.set) runs on a ThreadPool thread. Also, even if there are no timers in the app, if your I/O runs on threadpool threads, the issue is the same.

One solution for this would be to set the number of immediately available on-demand threads to a higher number using this before the loop:

// Get number of immediately available threads
ThreadPool.GetMinThreads(out var a, out var b);
Console.WriteLine($"worker         : " + a); // on my machine: 12
Console.WriteLine($"completion port: " + b); // on my machine: 12

// Set minimum number of immediately available threads.
// First parameter (worked threads) is relevant here:
ThreadPool.SetMinThreads(30, 12);

You can verify this by varying these numbers, or varying the upper bounds (= number of required threads) in your loops. It is not related to the type of Reset event, as in both cases working threads will block on Wait.

Replacing the timer by Thread.Sleep() allows to verify this as well. This does not run on ThreadPool; instead, it blocks the main thread:

Thread.Sleep(10000); // Block main thread for 10secs for demo purposes.
resetEvent.Set();

You can then see that 12 Threads execute immediately, then the next get added one by one every 500-1000s on my machine. After 10secs, event is set and threads are unblocked:

START
    Thread wait: 13 ms
    Thread wait: 13 ms
    Thread wait: 13 ms
    Thread wait: 13 ms
    Thread wait: 13 ms
        Sub Thread wait: 16 ms
        Sub Thread wait: 16 ms
        Sub Thread wait: 17 ms
        Sub Thread wait: 19 ms
        Sub Thread wait: 21 ms
        Sub Thread wait: 23 ms
        Sub Thread wait: 25 ms    // 12th thread
        Sub Thread wait: 1014 ms  // no more threads available, start adding them one by one
        Sub Thread wait: 2003 ms
        Sub Thread wait: 2517 ms
        Sub Thread wait: 3511 ms
        Sub Thread wait: 4518 ms
        Sub Thread wait: 5517 ms
        Sub Thread wait: 6512 ms
        Sub Thread wait: 7507 ms
Set event: 10013 ms               // set: all threads are released
        Sub Thread go  : 10013 ms
    Thread go  : 10013 ms
        Sub Thread go  : 10013 ms
        Sub Thread go  : 10013 ms

You can also set the Sleep() to 200ms instead of 10secs. You'll see that in this case, threads will run after about 200ms, because there will be no need to wait on the ThreadPool as with the Timer.

MS Docs:
SetMinThreads
Timer

evilmandarine
  • 4,241
  • 4
  • 17
  • 40