0

I got log message like this:

<pattern>
      %d{yyyy-MM-dd HH:mm:ss} %level %logger{0} %mdc - %msg %n
</pattern>

2018-03-22 13:17:47 INFO SomeController [X-Span-Export=false, method=someMethod(), X-B3-SpanId=deef3c47193ec4a6, X-B3-TraceId=0cecd4b78e1d8357] - Parameters: [id=40]

In %mdc I'm logging some data as you can see: span and trace id, method name and export.

I need to split the message into separate fields and use logstash to send via elastic search to kibana.

So I created grok filter:

  grok {
      match => { "message" => "%{DATESTAMP:date} %{WORD:level} %{WORD:class} \[X-Span-Export=%{GREEDYDATA:export}, method=%{GREEDYDATA:method}, X-B3-SpanId=%{GREEDYDATA:span_id}, X-B3-TraceId=%{GREEDYDATA:trace_id}\] - %{GREEDYDATA:log_message}" }
  }

The result:

{
  "date": [
    "18-03-22 13:17:47"
  ],
  "level": [
    "INFO"
  ],
  "class": [
    "SomeController"
  ],
  "export": [
    "false"
  ],
  "method": [
    "someMethod()"
  ],
  "span_id": [
    "deef3c47193ec4a6"
  ],
  "trace_id": [
    "0cecd4b78e1d8357"
  ],
  "log_message": [
    "Parameters: [id=40]"
  ]
}

This is working, but the problem is that %mdc is changing the positions, so log sometimes is like:

2018-03-22 13:17:47 INFO SomeController [X-B3-SpanId=deef3c47193ec4a6, method=someMethod(), X-Span-Export=false X-B3-TraceId=0cecd4b78e1d8357] - Parameters: [id=40]

As you can see here, first is spanId... so the filter is mixing the values. How can I make proper logstash config for this situation....

KiKo
  • 1,668
  • 7
  • 27
  • 56

0 Answers0