0

I have a use case where I need to log data to file Asynchronously which I do not see it possible with Microsoft.Extensions.Logging so need your help to figure out the best solution

  1. Use Seri ILogger interface, which according to this github doc does not seem to be using async await implementation but rather uses a background worker thread.

The wrapped sink (File in this case) will be invoked on a worker thread while your application's thread gets on with more important stuff.

  1. Write My Custom Logger which utilizes TextWriter.Synchronized and WriteLineAsync

OR if there is any other solution which I am not aware of

Mo Haidar
  • 3,748
  • 6
  • 37
  • 76
  • I don't understand the issue here: it's asynchronously logging to the file system via the File sink in Serilog. What about that doesn't meet your needs? What problem is it posing for you? I suspect you're having confusion about asynchronous vs async/await, and don't understand that async/await isn't actually what you want here. But it's hard to tell since you haven't clearly explain what you want and why you want it. – mason Oct 08 '22 at 14:49
  • A real Async Await Implementation would not go with a worker thread, see the below article. https://blog.stephencleary.com/2013/11/there-is-no-thread.html – Mo Haidar Oct 08 '22 at 14:52
  • You're confusing async/await with having an asynchronous process. Why do you think you need to make this use async/await? Please explain why you think that will be beneficial for your app. – mason Oct 08 '22 at 14:56
  • @mason You are right, not every asynchronous process needs to be implemented using Async Await, but in my case, I am building a load testing app where I need to reduce the execution time to the minimal, based on my understanding, every Log.Information will need to allocate a worker thread from the thread pool which is not optimal in my case – Mo Haidar Oct 08 '22 at 15:16
  • I would worry less about micro-optimizations (such as thread allocation) from an extremely battle tested library like Serilog, and instead focus on performance in my own code, if I had to choose where to expend my effort. This load testing app - does it run once and exit? Or does it continuously run? – mason Oct 08 '22 at 15:25
  • It is going to have both patterns, the client can choose to run it for X number of requests with X number of clients or to choose to run it for X number of minutes/hours with X number of clients – Mo Haidar Oct 08 '22 at 15:28
  • Okay. If you're generating a lot of logs and going for performance, you'll probably want to [enabled buffered writes](https://github.com/serilog/serilog-sinks-file#performance) (make sure your consider the implications if a batch fails to write) and make sure you Log.CloseAndFlush before your app exits to ensure all logs have finished writing before exiting, unless you're disposing of your logger in some other way. – mason Oct 08 '22 at 15:36

1 Answers1

3

It's normal for logging to use a background thread so it can expose a synchronous logging API. If all logging APIs were asynchronous, then that would force practically every method to be asynchronous.

Logging synchronously to a background thread also enables buffering and batched writes, which increase performance.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • Thanks for the answer, I got the first part, but regarding the second part, what would prevent an implantation of Async await of supporting buffering and batched writes? Excuse my ignorance – Mo Haidar Oct 08 '22 at 15:29
  • @MohamadHaidar: Yes, you could also do buffering and batched writes using `async`, although you may not want to. It would mean *most* of your log methods complete extremely quickly but then one semi-random one would take a long time. With the separate background thread, they all complete extremely quickly. – Stephen Cleary Oct 08 '22 at 16:14
  • Well, why they are gonna always be quick with background thread? If my understanding is correct, with every log.Information() call we need to allocate a worker thread from the thread pool, so what If for any reason I have to wail to get a thread allocated? Is not this a valid scenario? – Mo Haidar Oct 08 '22 at 17:47
  • No; there's only one background thread for the lifetime of the process. – Stephen Cleary Oct 08 '22 at 18:28
  • Thank you, @Stephan, now it is clear, I believe it is wise to use much mature library than creating mine, I have to deep dive to seriLog and master it. I created a simple custom Logger with Async Await implementation and find it faster than SeriLog Async implementation, is this because I am not using seriLog correctly? I understand that SeriLog is much mature, have great options, but would appreciate your words on what would I have done incorrectly in my comparison. There is a 20 second difference when it comes to 1M event. https://github.com/Mohammad-Moussa-Haidar/LoggersComparison – Mo Haidar Oct 08 '22 at 23:18
  • 1
    You might want to ask the Serilog team about that. – Stephen Cleary Oct 09 '22 at 02:17