2

I read this article and know something about IHttpAsyncHandler.

According to below picture,there are 2 threads created when I call an AsyncHandler.

When a request comes, IIS will grab a thread——Thread 1 to handle this request, and when it calls the beginXXX method, the thread 2 will be created and handle this real logic.

My questions are:

When Thread 2 is running,the connection is still stay,waiting for the response.

When Thread 2 is running, what's the status of Thread 1? Is it sleeping? Or it is released?

If the Thread 1 is sleeping, when Thread 2 is finished, Thread 1 is awoken and send the response??

If the Thread 1 is released, when Thread 2 is finished, is there a new thread created as Thread 1? And this new Thread 1 send the response to the client.

Thread 1 and Thread 2 are in the same Thread pool, aren't they ?? If so, the available threads count is balanced, what's the purpose to do like this?

enter image description here

roast_soul
  • 3,554
  • 7
  • 36
  • 73

1 Answers1

2

You are not thinking of the Thread Pool as a pool. When "Thread 1" finishes its work at the PreRender it is returned to the pool to be reused for any purpose. When the data is ready End will pick a random thread from the thread pool and finish its work and send the response on it. That random thread could be a new thread or it could be the same thread that did the work earlier that was returned to the pool.

The point of this is in the window between Begin and End that thread was retuned to the pool and could be servicing other connections. This allows you to have more concurrent connections than concurrent threads your system can handle, in the synchronous version once you hit the maximum number of concurrent threads you can no longer process new requests.

Let me go in to some detail of what is going on

Here is a timeline for a two synchronous connection request to show as a baseline, for simplicity sake we will say every step takes 1ms to execute except PreRender which takes 19ms to complete.

╔═══════╦════════════════════════╦══════════════════╦═════════════════╦═══════════════════╗
║ Time  ║ Action (Connection #)  ║ Open Connections ║ Running Threads ║ Available Threads ║
╠═══════╬════════════════════════╬══════════════════╬═════════════════╬═══════════════════╣
║  0 ms ║ Connection Request (1) ║                0 ║               0 ║                 2 ║
║  1 ms ║ PreInit  (1)           ║                1 ║               1 ║                 1 ║
║  2 ms ║ Init  (1)              ║                1 ║               1 ║                 1 ║
║  3 ms ║ InitComplete  (1)      ║                1 ║               1 ║                 1 ║
║  4 ms ║ PreLoad  (1)           ║                1 ║               1 ║                 1 ║
║  5 ms ║ LoadComplete  (1)      ║                1 ║               1 ║                 1 ║
║  6 ms ║ PreRender  (1)         ║                1 ║               1 ║                 1 ║
║ 10 ms ║ Connection Request (2) ║                1 ║               1 ║                 1 ║
║ 11 ms ║ PreInit  (2)           ║                2 ║               2 ║                 0 ║
║ 12 ms ║ Init  (2)              ║                2 ║               2 ║                 0 ║
║ 13 ms ║ InitComplete  (2)      ║                2 ║               2 ║                 0 ║
║ 14 ms ║ PreLoad  (2)           ║                2 ║               2 ║                 0 ║
║ 15 ms ║ LoadComplete  (2)      ║                2 ║               2 ║                 0 ║
║ 16 ms ║ PreRender  (2)         ║                2 ║               2 ║                 0 ║
║ 25 ms ║ PreRenderComplete  (1) ║                2 ║               2 ║                 0 ║
║ 26 ms ║ SaveState  (1)         ║                2 ║               2 ║                 0 ║
║ 27 ms ║ SaveStateComplete  (1) ║                2 ║               2 ║                 0 ║
║ 28 ms ║ Render  (1)            ║                2 ║               2 ║                 0 ║
║ 29 ms ║ Send Response  (1)     ║                1 ║               1 ║                 1 ║
║ 35 ms ║ PreRenderComplete  (2) ║                1 ║               1 ║                 1 ║
║ 36 ms ║ SaveState  (2)         ║                1 ║               1 ║                 1 ║
║ 37 ms ║ SaveStateComplete  (2) ║                1 ║               1 ║                 1 ║
║ 38 ms ║ Render  (2)            ║                1 ║               1 ║                 1 ║
║ 39 ms ║ Send Response  (2)     ║                0 ║               0 ║                 2 ║
╚═══════╩════════════════════════╩══════════════════╩═════════════════╩═══════════════════╝

And here is the timeline for the async version.

╔═══════╦════════════════════════╦══════════════════╦═════════════════╦═══════════════════╗
║ Time  ║ Action (Connection #)  ║ Open Connections ║ Running Threads ║ Available Threads ║
╠═══════╬════════════════════════╬══════════════════╬═════════════════╬═══════════════════╣
║  0 ms ║ Connection Request (1) ║                0 ║               0 ║                 2 ║
║  1 ms ║ PreInit  (1)           ║                1 ║               1 ║                 1 ║
║  2 ms ║ Init  (1)              ║                1 ║               1 ║                 1 ║
║  3 ms ║ InitComplete  (1)      ║                1 ║               1 ║                 1 ║
║  4 ms ║ PreLoad  (1)           ║                1 ║               1 ║                 1 ║
║  5 ms ║ LoadComplete  (1)      ║                1 ║               1 ║                 1 ║
║  6 ms ║ PreRender  (1)         ║                1 ║               1 ║                 1 ║
║  7 ms ║ Begin (1)              ║                1 ║               0 ║                 2 ║
║ 10 ms ║ Connection Request (2) ║                1 ║               0 ║                 2 ║
║ 11 ms ║ PreInit  (2)           ║                2 ║               1 ║                 1 ║
║ 12 ms ║ Init  (2)              ║                2 ║               1 ║                 1 ║
║ 13 ms ║ InitComplete  (2)      ║                2 ║               1 ║                 1 ║
║ 14 ms ║ PreLoad  (2)           ║                2 ║               1 ║                 1 ║
║ 15 ms ║ LoadComplete  (2)      ║                2 ║               1 ║                 1 ║
║ 16 ms ║ PreRender  (2)         ║                2 ║               1 ║                 1 ║
║ 17 ms ║ Begin (2)              ║                2 ║               0 ║                 2 ║
║ 25 ms ║ End (1)                ║                2 ║               1 ║                 1 ║
║ 26 ms ║ PreRenderComplete  (1) ║                2 ║               1 ║                 1 ║
║ 27 ms ║ SaveState  (1)         ║                2 ║               1 ║                 1 ║
║ 28 ms ║ SaveStateComplete  (1) ║                2 ║               1 ║                 1 ║
║ 29 ms ║ Render  (1)            ║                2 ║               1 ║                 1 ║
║ 30 ms ║ Send Response  (1)     ║                1 ║               0 ║                 2 ║
║ 35 ms ║ End (2)                ║                1 ║               1 ║                 1 ║
║ 36 ms ║ PreRenderComplete  (2) ║                1 ║               1 ║                 1 ║
║ 37 ms ║ SaveState  (2)         ║                1 ║               1 ║                 1 ║
║ 38 ms ║ SaveStateComplete  (2) ║                1 ║               1 ║                 1 ║
║ 39 ms ║ Render  (2)            ║                1 ║               1 ║                 1 ║
║ 40 ms ║ Send Response  (2)     ║                0 ║               0 ║                 2 ║
╚═══════╩════════════════════════╩══════════════════╩═════════════════╩═══════════════════╝

Now when dealing with two or less connections there is no real benefit for asynchronous, in fact it might even be slightly slower due to the additional overhead.

However see what happens when we have more than 2 concurrent connections.

Sync:

╔═══════╦════════════════════════╦══════════════════╦═════════════════╦═══════════════════╗
║ Time  ║ Action (Connection #)  ║ Open Connections ║ Running Threads ║ Available Threads ║
╠═══════╬════════════════════════╬══════════════════╬═════════════════╬═══════════════════╣
║  0 ms ║ Connection Request (1) ║                0 ║               0 ║                 2 ║
║  1 ms ║ PreInit  (1)           ║                1 ║               1 ║                 1 ║
║  2 ms ║ Init  (1)              ║                1 ║               1 ║                 1 ║
║  3 ms ║ InitComplete  (1)      ║                1 ║               1 ║                 1 ║
║  4 ms ║ PreLoad  (1)           ║                1 ║               1 ║                 1 ║
║  5 ms ║ LoadComplete  (1)      ║                1 ║               1 ║                 1 ║
║  6 ms ║ PreRender  (1)         ║                1 ║               1 ║                 1 ║
║ 10 ms ║ Connection Request (2) ║                1 ║               1 ║                 1 ║
║ 11 ms ║ PreInit  (2)           ║                2 ║               2 ║                 0 ║
║ 12 ms ║ Init  (2)              ║                2 ║               2 ║                 0 ║
║ 13 ms ║ InitComplete  (2)      ║                2 ║               2 ║                 0 ║
║ 14 ms ║ PreLoad  (2)           ║                2 ║               2 ║                 0 ║
║ 15 ms ║ LoadComplete  (2)      ║                2 ║               2 ║                 0 ║
║ 16 ms ║ PreRender  (2)         ║                2 ║               2 ║                 0 ║
║ 20 ms ║ Connection Request (3) ║                2 ║               2 ║                 0 ║
║ 25 ms ║ PreRenderComplete  (1) ║                2 ║               2 ║                 0 ║
║ 26 ms ║ SaveState  (1)         ║                2 ║               2 ║                 0 ║
║ 27 ms ║ SaveStateComplete  (1) ║                2 ║               2 ║                 0 ║
║ 28 ms ║ Render  (1)            ║                2 ║               2 ║                 0 ║
║ 29 ms ║ Send Response  (1)     ║                1 ║               1 ║                 1 ║
║ 30 ms ║ PreInit  (3)           ║                2 ║               2 ║                 0 ║
║ 31 ms ║ Init  (3)              ║                2 ║               2 ║                 0 ║
║ 32 ms ║ InitComplete  (3)      ║                2 ║               2 ║                 0 ║
║ 33 ms ║ PreLoad  (3)           ║                2 ║               2 ║                 0 ║
║ 34 ms ║ LoadComplete  (3)      ║                2 ║               2 ║                 0 ║
║ 35 ms ║ PreRender  (3)         ║                2 ║               2 ║                 0 ║
║ 35 ms ║ PreRenderComplete  (2) ║                2 ║               2 ║                 0 ║
║ 36 ms ║ SaveState  (2)         ║                2 ║               2 ║                 0 ║
║ 37 ms ║ SaveStateComplete  (2) ║                2 ║               2 ║                 0 ║
║ 38 ms ║ Render  (2)            ║                2 ║               2 ║                 0 ║
║ 39 ms ║ Send Response  (2)     ║                1 ║               1 ║                 1 ║
║ 54 ms ║ PreRenderComplete  (3) ║                1 ║               1 ║                 1 ║
║ 55 ms ║ SaveState  (3)         ║                1 ║               1 ║                 1 ║
║ 56 ms ║ SaveStateComplete  (3) ║                1 ║               1 ║                 1 ║
║ 57 ms ║ Render  (3)            ║                1 ║               1 ║                 1 ║
║ 58 ms ║ Send Response  (3)     ║                0 ║               0 ║                 2 ║
╚═══════╩════════════════════════╩══════════════════╩═════════════════╩═══════════════════╝

Async:

╔═══════╦════════════════════════╦══════════════════╦═════════════════╦═══════════════════╗
║ Time  ║ Action (Connection #)  ║ Open Connections ║ Running Threads ║ Available Threads ║
╠═══════╬════════════════════════╬══════════════════╬═════════════════╬═══════════════════╣
║  0 ms ║ Connection Request (1) ║                0 ║               0 ║                 2 ║
║  1 ms ║ PreInit  (1)           ║                1 ║               1 ║                 1 ║
║  2 ms ║ Init  (1)              ║                1 ║               1 ║                 1 ║
║  3 ms ║ InitComplete  (1)      ║                1 ║               1 ║                 1 ║
║  4 ms ║ PreLoad  (1)           ║                1 ║               1 ║                 1 ║
║  5 ms ║ LoadComplete  (1)      ║                1 ║               1 ║                 1 ║
║  6 ms ║ PreRender  (1)         ║                1 ║               1 ║                 1 ║
║  7 ms ║ Begin (1)              ║                1 ║               0 ║                 2 ║
║ 10 ms ║ Connection Request (2) ║                1 ║               0 ║                 2 ║
║ 11 ms ║ PreInit  (2)           ║                2 ║               1 ║                 1 ║
║ 12 ms ║ Init  (2)              ║                2 ║               1 ║                 1 ║
║ 13 ms ║ InitComplete  (2)      ║                2 ║               1 ║                 1 ║
║ 14 ms ║ PreLoad  (2)           ║                2 ║               1 ║                 1 ║
║ 15 ms ║ LoadComplete  (2)      ║                2 ║               1 ║                 1 ║
║ 16 ms ║ PreRender  (2)         ║                2 ║               1 ║                 1 ║
║ 17 ms ║ Begin (2)              ║                2 ║               0 ║                 2 ║
║ 20 ms ║ Connection Request (3) ║                3 ║               0 ║                 2 ║
║ 21 ms ║ PreInit  (3)           ║                3 ║               1 ║                 1 ║
║ 22 ms ║ Init  (3)              ║                3 ║               1 ║                 1 ║
║ 23 ms ║ InitComplete  (3)      ║                3 ║               1 ║                 1 ║
║ 24 ms ║ PreLoad  (3)           ║                3 ║               1 ║                 1 ║
║ 25 ms ║ End (1)                ║                3 ║               2 ║                 0 ║
║ 25 ms ║ LoadComplete  (3)      ║                3 ║               2 ║                 0 ║
║ 26 ms ║ PreRenderComplete  (1) ║                3 ║               2 ║                 0 ║
║ 26 ms ║ PreRender  (3)         ║                3 ║               2 ║                 0 ║
║ 27 ms ║ SaveState  (1)         ║                3 ║               2 ║                 0 ║
║ 27 ms ║ Begin (3)              ║                3 ║               1 ║                 0 ║
║ 28 ms ║ SaveStateComplete  (1) ║                3 ║               1 ║                 1 ║
║ 29 ms ║ Render  (1)            ║                3 ║               1 ║                 1 ║
║ 30 ms ║ Send Response  (1)     ║                2 ║               0 ║                 2 ║
║ 35 ms ║ End (2)                ║                2 ║               1 ║                 1 ║
║ 36 ms ║ PreRenderComplete  (2) ║                2 ║               1 ║                 1 ║
║ 37 ms ║ SaveState  (2)         ║                2 ║               1 ║                 1 ║
║ 38 ms ║ SaveStateComplete  (2) ║                2 ║               1 ║                 1 ║
║ 39 ms ║ Render  (2)            ║                2 ║               1 ║                 1 ║
║ 40 ms ║ Send Response  (2)     ║                1 ║               0 ║                 2 ║
║ 45 ms ║ End (3)                ║                1 ║               1 ║                 1 ║
║ 46 ms ║ PreRenderComplete  (3) ║                1 ║               1 ║                 1 ║
║ 47 ms ║ SaveState  (3)         ║                1 ║               1 ║                 1 ║
║ 48 ms ║ SaveStateComplete  (3) ║                1 ║               1 ║                 1 ║
║ 49 ms ║ Render  (3)            ║                1 ║               1 ║                 1 ║
║ 50 ms ║ Send Response  (3)     ║                0 ║               0 ║                 2 ║
╚═══════╩════════════════════════╩══════════════════╩═════════════════╩═══════════════════╝

Notice that in the sync version connection 3 came in at 20ms but it had to wait until 30ms when a thread became available to process the request. In contrast the async version gave the threads back in to the pool while it was waiting for PreRender to finish so connection 3 was immediately able to start processing.

So using async does not increase Transactions Per Second, in fact it will likely slightly lower it. However what it does do is increase your "Max Concurrent Transaction" count and increase your total throughput.

Scott Chamberlain
  • 124,994
  • 33
  • 282
  • 431
  • first,are Thread 1 and Thread 2 in the same thread pool? If so, releasing Thread 1 and creating Thread 2 are meaningless. Because the quantity of available threads is same. – roast_soul Aug 05 '14 at 15:24
  • @roast_soul Between `Begin` and `End` the quantity of available threads go up by one. So yes, you start and end with the same number of threads, but half way through you "lend out" your thread while you are waiting for work to complete and someone else could use it while you are waiting. In the synchronous model no one can use your thread while you are waiting. – Scott Chamberlain Aug 05 '14 at 16:03
  • you means that thread 1 and thread 2 are two different kinds. Such as Thread 1 is a worker thread just used to process request, and Thread 2 is a I/O thread just used to callback.Is it? – roast_soul Aug 05 '14 at 16:31
  • As thread 1 is for request, so it's precious, letting thread 1 release asap can increase the TPS(Transactions Per Second). Am I right? – roast_soul Aug 05 '14 at 16:36
  • @roast_soul see my expanded answer. It slows down the TPS of a single pipleline but allows you to have more pipelines at once. – Scott Chamberlain Aug 05 '14 at 18:07