11

I using Serilog inside my aspnet core app for logging. And i need to write log events to console pretty frequently (300-500 events per second). I run my app inside docker container and procces console logs using orchestrator tools.

So my question: should i use Async wrapper for my Console sink and will i get any benefits from that? I read the documentation (https://github.com/serilog/serilog-sinks-async), but can't understand is it actual for Console sink or not.

Frank59
  • 3,141
  • 4
  • 31
  • 53

1 Answers1

15

The Async Sink takes the already-captured LogEvent items and shifts them to a single background processor from multiple foreground threads using a ConcurrentQueue Producer/Consumer collection. In general that's a good thing for stable throughput esp with that throughput of events.

Also if sending to >1 sink, shifting to a background thread which will be scheduled as necessary focusing on that workload (i.e., paths propagating to sinks being in cache) can be good if you have enough cores available and/or Sinks block even momentarily.

Having said that, to base anything of this information is premature optimization.

Console sinks and their ability to ingest efficiently without blocking if you don't put an Async in front, always Depends a lot - for example, hosting environments typically synthesize a stdout that buffers efficiently. When that works well, adding an Async in front of the Console sink is merely going to prolong object lifetimes without much benefit vs allowing each thread submit to the Console sink directly.

So, it depends - IME feeding everything to Async and doing all processing there (e.g. writing to a buffered file, emitting every .5s (perhaps to a sidecar process that forwards to your log store)) can work well. The bottom line is that a good load generator rig is a very useful thing for any high throughput app. Once you have one, you can experiment - I've seen 30% throughput gains from reorganizing the exact same output and how it's scheduled (admittedly I also switched to Serilog during that transition - you're unlikely to see anything of that order).

Ruben Bartelink
  • 59,778
  • 26
  • 187
  • 249
  • 2
    Thank you for detailed answer! – Frank59 Dec 26 '18 at 19:01
  • 2
    Very good [blog post from Rick Strahl which covers lots of this ground esp in the context of ASP.NET Core](https://weblog.west-wind.com/posts/2018/Dec/31/Dont-let-ASPNET-Core-Default-Console-Logging-Slow-your-App-down) - doesn't change the fundamental point I'm trying to get across above: doing the legwork to make it possible to measure is key. – Ruben Bartelink Jan 02 '19 at 11:13
  • 1
    When i tested serilog file vs console logging performance, i also disabled standard netcore logging first. It increased performance of test app near 2-3 times. – Frank59 Jan 09 '19 at 11:20