Does await Task.Delay; really enable web server to process more simultaneous requests?

Question

From Pro Asynchrnous Programming with .Net:

    for (int nTry = 0; nTry < 3; nTry++)
    {
        try
        {
            AttemptOperation();
            break;
        }
        catch (OperationFailedException) { }
        Thread.Sleep(2000);
    }
While sleeping, the thread doesn’t consume any CPU-based resources, but the fact that the thread is alive means that it is still consuming memory resources. On a desktop application this is probably no big deal, but on a server application, having lots of threads sleeping is not ideal because if more work arrives on the server, it may have to spin up more threads, increasing memory pressure and adding additional resources for the OS to manage.

Ideally, instead of putting the thread to sleep, you would like to simply give it up, allowing the thread to be free to serve other requests. When you are ready to continue using CPU resources again, you can obtain a thread ( not necessarily the same one ) and continue processing. You can solve this problem by not putting the thread to sleep, but rather using await on a Task that is deemed to be completed in a given period.
    for (int nTry = 0; nTry < 3; nTry++)
    {
        try
        {
            AttemptOperation();
            break;
        }
        catch (OperationFailedException) { }
        await Task.Delay(2000);
    }

I don't follow author's reasoning. While it's true that calling await Task.Delay will release this thread ( which is processing a request ), but it's also true that task created by Task.Delay will occupy some other thread to run on. So does this code really enable server to process more simultaneous requests or is the text wrong?!

usr · Accepted Answer · 2014-05-29T18:37:24.603

Task.Delay does not occupy some other thread. It gives you a task without blocking. It starts a timer that completes that task in its callback. The timer does not use any thread while waiting.

It is a common myth that async actions like delays or IO just push work to a different thread. They do not. They use OS facilities to truly use zero threads while the operation is in progress. (They obviously need to use some thread to initiate and complete the operation.)

If async was just pushing work to a different thread it would be mostly useless. It's value would be just to keep the UI responsive in client apps. On the server it would only cause harm. It is not so.

The value of async IO is to reduce memory usag (less thread stacks), context switching and thread-pool utilization.

The async version of the code you posted would scale to literally tens of thousands of concurrent requests (if you increase the ASP.NET limits appropriately, which is a simple web.config change) with small memory usage.

Does await Task.Delay; really enable web server to process more simultaneous requests?

1 Answers1