0

Recently, one of our most senior engineers asked me about context switching with respect to using syslog-ng vs. writing logs out from our application to disk.

Context: I want to use syslog-ng to log & ship output from our application written in C++ to logstash on a remote logserver host, then shove it all into elasticsearch & use Kibana as a front-end for log viewing, analysis, and derivation of useful metrics. ELK stack

We currently utilize an FSYNC buffer of either 4K or 8K that spits logs out in intervals to the logfile on disk; that is to say, we're not forcing a write to disk for each log entry.

Like any good performance-seeking engineer, he wants to understand whether we'll see greater context switching or if we can make any performance gains by leveraging syslog-ng.

So the question is: Will using syslog-ng reduce or increase context switches on that application's host.

My expertise leads me here, to ask the question - given that I don't have sufficient knowledge to come up with the answer on my own.

Long-time lurker, still new to posting. Thanks!

Armen R.
  • 1
  • 3

1 Answers1

1

It depends on how many logs you have to handle and how much resources you're willing to dedicate.

We use Kibana/Elastic search with Logstash and the load does get pretty heavy, but then again we have over 400 servers so YMMW. Java isn't really known for being lenient on resources either. However, on the plus side it's fairly easy to set up.

Parsing the logs in Logstash can be done with grok. Throw up a couple of VMs and play around with it, if you have a large environment fine-tuning configuration is a must to make it bearable, or if you have money you can throw hardware at it until it behaves.

Tobias Järvelöv
  • 167
  • 1
  • 7
  • Our application has its own dedicated host. Logs are FSYNC'ed out of the app onto logs on disk. I'm asking about whether using syslog-ng will reduce or increase context switches on that host. – Armen R. Sep 11 '14 at 23:52