0

I'm currently in front of a structure problem with logstash. I have a syslog-ng client sending logs from different files through the network to an ELK stack.

I noticed that Logstash is mixing logs, especially multiline with adding exception lines to non error logs from others files. So, i guess the trouble is that my logs are treated without any differentiation from their file origins. I found 2 ways to avoid that, but they are not optimal in my opinion:

  • Instead of using syslog-ng, use FileBeat and add a tag for each file which identify their origins. Then, parse my log with Logstash depending of this tag. The problem is that using syslog-ng as client is a requirement, and it will bother me if I have to change it
  • Change my syslog-ng sources to send each log files to a different port on ELK. I found that a little dirty and can be embarrassing with a great quantity of log files

What do you think about that ? Did I miss a better solution? Is there a way to add a tag just like filebeat in syslog-ng?

Jason Aller
  • 3,541
  • 28
  • 38
  • 38
agentn
  • 33
  • 1
  • 5
  • For the filebeat solution, filebeat can deal with the multiline issues, which would avoid having to add tags. – baudsp May 17 '19 at 11:36
  • syslog-ng can send logs to Elastic directly, if the problem is in logstash, circumventing logstash might be a solution. See http://support.oneidentity.com/technical-documents/syslog-ng-open-source-edition/3.21/administration-guide/destination-forward-send-and-store-log-messages/elasticsearch-http-sending-messages-to-elasticsearch-http-bulk-api – Robert Fekete May 17 '19 at 14:44
  • Doesn't syslog-ng allows you to add a tag to each message? You could use it or the filename to filter on your logstash. – leandrojmp May 18 '19 at 13:10
  • @RobertFekete yes but removing Logstash is not a solution, parsing is required :) – agentn May 22 '19 at 08:09
  • @leandrojmp I tried to add tags to my sources but then I cannot find a way to identify them in my input and parse in function of these tags. Syslog-ng option : https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.17/administration-guide/tags – agentn May 22 '19 at 08:14
  • Does the tags appears on the message that logstash receives? Can you provide a sample message? – leandrojmp May 23 '19 at 00:48
  • Indeed, it doesn't appear on the log msg, here is a sample from the logstash debug output : ' logstash_1 | [2019-05-23T07:17:34,702][DEBUG][logstash.codecs.multiline] Multiline {:pattern=>"(^%{PATTERN_DATE})", :text=>"<13>May 23 07:18:07 9186c295af3e 23-05-19 09:00:57 WARN [BasicResourcePool] Having failed to acquire a resource, com.mchange.v2.resourcepool.BasicResourcePool@5d7f6f39 is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.", :match=>false, :negate=>true} ' – agentn May 23 '19 at 07:26

0 Answers0