Benchmarking ASP.NET concurrent requests poor results

Question

I have the following code that I benchmark with jMeter and get about 3000 request per second on my localhost machine(the await is missing intentionally to run synchronously):

public async Task<HttpResponseMessage> Get()
{
    var resp = new HttpResponseMessage(HttpStatusCode.OK);
    resp.Content = new StringContent(Thread.CurrentThread.ManagedThreadId.ToString(), Encoding.UTF8, "text/plain");
    return resp;
}

The problem is that when I pause the request for one second like below, for some reason the throughput is down to 10 requests per second for each w3wp.exe process (again the await is missing intentionally to run synchronously):

public async Task<HttpResponseMessage> Get()
    {
        Task.Delay(1000).Wait();
        var resp = new HttpResponseMessage(HttpStatusCode.OK);
        resp.Content = new StringContent(Thread.CurrentThread.ManagedThreadId.ToString(), Encoding.UTF8, "text/plain");
        return resp;
    }

Even when I do use await there is no difference and the 10 requests per second does not improve at all:

public async Task<HttpResponseMessage> Get()
{
    await Task.Delay(1000);
    var resp = new HttpResponseMessage(HttpStatusCode.OK);
    resp.Content = new StringContent(Thread.CurrentThread.ManagedThreadId.ToString(), Encoding.UTF8, "text/plain");
    return resp;
}

I tried all the config settings and nothing makes any change at all: `

web.config

  <system.net>
    <connectionManagement>
      <add address="*" maxconnection="65400" />
    </connectionManagement>
  </system.net>

aspnet.config

  <system.web>
    <applicationPool 
        maxConcurrentThreadsPerCPU="100" />
  </system.web>

machine.config

 <processModel
 autoConfig="false"
 memoryLimit="70"
 maxWorkerThreads="100"
 maxIoThreads="100" />

The configs are set for both x86 and x64

I have 32 gigs of mem and 4 physical cores, Windows 10.

The CPU doesn't go over 10% load when benching the 10 requests per second.

The above code uses WEB API, but of course I reproduce the same results using a HTTP Handler.

Using a single general purpose webserver a well designed ASP.NET application will process around 300 req/s of real user interaction. Poorly architected systems start seeing those zeros disappear. To increase throughput on a single windows server you want to look at using "web gardens". In general your bottleneck is not your webserver but your shared state (e.g. your database). — Chris Marisic, Dec 10 '15 at 00:49
10 req/s for a request execution duration of 1 sec seems pretty normal with any technology. And it's also normal that the CPU is not used (requests just wait without consuming any processing). What do you expect? — Simon Mourier, Dec 13 '15 at 09:54

score 2 · Answer 1 · answered Dec 18 '15 at 20:01

Here's a possible understanding. One to investigate, anyway.

Task.Delay() creates a new task, whose job is to pause. If I understand right, tasks often get dispatched to the .Net worker pool, which has a limited size. (You can check with ThreadPool.GetMaxThreads) When you try to put too much in, code will 'back up' as it waits for the thread pool to have space.

So let's say you have a thread pool of size 40. Once you've dispatched 40 tasks, all waiting a second, you max out the thread pool. Your bottleneck would be the tasks, gumming up the thread pool, not yielding space.

Normally, tasks that do expensive IO like database queries or file IO yield control while they wait for the work to be done. I wonder if Task.Delay is more 'clingy'.

Try swapping Task.Delay() for System.Threading.Thread.Sleep() and see if that changes anything.

Using System.Threading.Thread.Sleep() dropped the requests per second even more to 8 requests per second. — realPro, Dec 18 '15 at 20:33

score 1 · Answer 2 · answered Jan 14 '16 at 17:46

1

I know for Windows 8 there is a maximum concurrent connection limit of 10 to stop people trying to use consumer OS's to run server workloads. I see no reason why Windows 10 would be any different.

http://blogs.iis.net/owscott/windows-8-iis-8-concurrent-requests-limit

answered Jan 14 '16 at 17:46

Joe U

366
3
12

Yes, I saw this post too and so installed server 2016 to run the tests, but the behavior did not change in any way at all. – realPro Jan 14 '16 at 19:00

Benchmarking ASP.NET concurrent requests poor results

2 Answers2