3

For a demo I intend to run a benchmark where I compare .NET Core running on Kestrel, with .NET Framework 4.6.1 running on IIS. Both on local machine.

"The Internet" says that Kestrel is much faster, but in my own benchmark there's no notable difference to IIS. And most of the time IIS is even faster. How is that?

I have the "same" code running on .NET Framework (using EF) and .NET Core (using EFCore). The controller looks like this. It will write something to DB, and then fetch it and return it. I close the DBContext to make sure there's no cache. Code for .NET Core below (code for .NET Framework is similiar except no DI).

[Route("api/[controller]")]
public class ValuesController : Controller
{
    private readonly DbContextOptions<DemoContext> options;

    public ValuesController(DbContextOptionsBuilder<DemoContext> builder)
    {
        this.options = builder.Options;
    }

    // GET api/values
    [HttpGet]
    public async Task<IActionResult> Get()
    {
        var id = Guid.NewGuid();
        using (var context = new DemoContext(options))
        {

            var newItem = new DemoTable()
            {
                Id = id,
                Stamp = DateTime.Now,
                System = "Core"
            };
            context.DemoTable.Add(newItem);
            await context.SaveChangesAsync();
        }

        using (var context = new DemoContext(options))
        {
            var item = await context.DemoTable.SingleAsync(x => x.Id == id);
            return Ok(item);

        }
    }
}

My Program.cs looks like this:

public static IWebHost BuildWebHost(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .UseKestrel(options =>
        {
            options.Limits.MaxConcurrentConnections = 1000;
            options.Listen(IPAddress.Loopback, 5050);
        })
        .ConfigureLogging((context, logging) =>
        {
            logging.ClearProviders();
        })
        .UseStartup<Startup>()
        .Build();

I run it with dotnet as dotnet demoApp.dll.

I test it with an application which makes a number of requests.

    var cts = new CancellationTokenSource();
    int processed = 0;
    int seconds = 30;

    var url = new Uri($"http://localhost:{port}");
    var tasks = new Task[20];
    for (int i = 0; i < tasks.Length; i++)
    {
        tasks[i] = Task.Factory.StartNew(() =>
        {
            while (!token.IsCancellationRequested)
            {
                var client = new RestClient(url);
                var request = new RestRequest("/api/values");
                client.ExecuteAsyncGet(request, (response, handle) =>
                {
                    if(!token.IsCancellationRequested)
                        Interlocked.Add(ref processed, 1);
                }, "GET");
            }
        });
    }
    Thread.Sleep(seconds * 1000);
    cts.Cancel();

I get more requests when I run it against .NET Framework with IIS than when I run it against Core with Kestrel.

I've tried changing options.Limits in Kestrel with not success. My assumption is that there's something wrong with my "benchmarking application" or that my local machine is the bottleneck itself.

Why is IIS handling the requests much faster than Kestrel?

UPDATE:

If I remove EF and only return OK() in the controller IIS still performs better.

I wake the servers up before benchmarking.

I build it as Release.

When I run with 10 threads for 30 seconds IIS will handle 600 request and Kestrel 300 requests.

gcServer is set to true.

smoksnes
  • 10,509
  • 4
  • 49
  • 74
  • 1
    You might be testing EF Core vs EF perfomance for example, you cannot figure out with such test. If you are testing Kestel vs IIS - you have to exclude all unrelated factors. – Evk May 17 '18 at 08:14
  • @Evk - I actually tested it without EF first. But with the same result. – smoksnes May 17 '18 at 08:16
  • 1
    We'd a similar observation in past for a dotnet core application. Based on some recommendation, we have enabled the gcServer setting to true which have increased the request per second. You can read it about here --- https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcserver-element – user1672994 May 17 '18 at 08:25
  • @user1672994 - Thx. I've tried both with no luck. – smoksnes May 17 '18 at 08:39
  • Just how big a difference are we talking about here? Is it a significant amount of time to a human being? – Ryan Lundy Jul 02 '18 at 08:20

1 Answers1

1

You cant test a web hosts performance by connecting it to a db. The db or your business logic will be the bottleneck .

Start with a clean project or better yet just grab the benchmark. For high end performance many things come into play ..

The fact you are handling 300 or 600 requests ( pitiful) suggest something else in the pipeline.. Note your controller is injecting a db context on each request as well + whatever middleware you have.

user1496062
  • 1,309
  • 1
  • 7
  • 22
  • Thanks for your answer, but as mentioned in the comments I've tried it without Db as well. Same result. There is no other middleware than MVC involved. – smoksnes Jul 03 '18 at 10:28
  • Did you do a clean project ? The code you posted still does db calls even if you don't talk to the db via the constructor injection... 300 or 600 request tells you something is wrong / different.. You can create a new project run 10 clients and get 10 to 100* more returning a static Ok. – user1496062 Jul 09 '18 at 00:18
  • yes. Did a clean project first. Added db later. No middleware except MVC. – smoksnes Jul 09 '18 at 12:17
  • Are you using the stupid IIS integration that's default on some projects ? Sorry I missed this reply , but I would just grab a performance sample and start with that / see how my code is different but I can tell you its fast out of the box for many projects that said most of them are core not net 461. – user1496062 Oct 12 '18 at 02:52
  • nope. No IIS integration. But I believe you are right. I should just grab some other benchmark project and go on from there... – smoksnes Oct 12 '18 at 10:45