I got log message like this:
<pattern>
%d{yyyy-MM-dd HH:mm:ss} %level %logger{0} %mdc - %msg %n
</pattern>
2018-03-22 13:17:47 INFO SomeController [X-Span-Export=false, method=someMethod(), X-B3-SpanId=deef3c47193ec4a6, X-B3-TraceId=0cecd4b78e1d8357] - Parameters: [id=40]
In %mdc I'm logging some data as you can see: span and trace id, method name and export.
I need to split the message into separate fields and use logstash to send via elastic search to kibana.
So I created grok filter:
grok {
match => { "message" => "%{DATESTAMP:date} %{WORD:level} %{WORD:class} \[X-Span-Export=%{GREEDYDATA:export}, method=%{GREEDYDATA:method}, X-B3-SpanId=%{GREEDYDATA:span_id}, X-B3-TraceId=%{GREEDYDATA:trace_id}\] - %{GREEDYDATA:log_message}" }
}
The result:
{
"date": [
"18-03-22 13:17:47"
],
"level": [
"INFO"
],
"class": [
"SomeController"
],
"export": [
"false"
],
"method": [
"someMethod()"
],
"span_id": [
"deef3c47193ec4a6"
],
"trace_id": [
"0cecd4b78e1d8357"
],
"log_message": [
"Parameters: [id=40]"
]
}
This is working, but the problem is that %mdc is changing the positions, so log sometimes is like:
2018-03-22 13:17:47 INFO SomeController [X-B3-SpanId=deef3c47193ec4a6, method=someMethod(), X-Span-Export=false X-B3-TraceId=0cecd4b78e1d8357] - Parameters: [id=40]
As you can see here, first is spanId... so the filter is mixing the values. How can I make proper logstash config for this situation....