1

Context

I have the Red Hat OpenShift Logging Operator installed in a cluster, explained on this page. This is basically an EFK stack (ElasticSearch, Fluentd, Kibana). Normally with Fluentd you can configure many plugins and hundreds of parameters, but as you can see on this OpenShift page there are only 9 Fluentd parameters available (all chunk-related) because the logging stack is packaged as a Red Hat Operator. Adding custom configuration is not an option for me because it is unsupported and won't receive updates from RedHat.

Problem

When an application produces a stack trace after an error, Kibana shows 1 line of the stack trace per record . I need the whole stack trace in 1 record. This is possible with the Fluentd multiline plugin but this is not a supported parameter with the Red Hat OpenShift Logging Operator (which uses Fluentd).

What are my options for grouping stack trace lines into one Kibana record, aside from installing a new/standalone logging stack?

M455
  • 15
  • 5

1 Answers1

0

There seems to be a supported solution now: Multiline logs in OpenShift 4 with ClusterLogForwarder API

ciis0
  • 311
  • 1
  • 9
  • this applies to all multiline, or just multiline with errors? – sloweriang Oct 20 '22 at 05:54
  • @sloweriang If I understand you correctly, just multiline with errors. As far as i can remember `detectMultilineErrors` under-the-hood adds [fluent-plugin-detect-exceptions](https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions). The list of patterns is in [`exception_detector.rb`](https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/17c846bc7e6f58f26794684cca9bd306d2f857cf/lib/fluent/plugin/exception_detector.rb). – ciis0 Oct 20 '22 at 09:19
  • so if I want not just multiline with errors, but also other multiline(e.g http post) , can openshift do it? – sloweriang Oct 21 '22 at 04:30
  • @sloweriang I think unfortunately this is not possible, no. – ciis0 Oct 21 '22 at 08:37
  • For the applications(s) my team is running we're currently migrating to JSON-Logging because some the plugin is picky what counts as an exception -- for example including correlation IDs in the log format lines breaks detection... – ciis0 Oct 21 '22 at 08:39