My app writes logs as json to stdout. App runs inside docker. And docker logs stdout also in json format. So I have a json nested in json. Here is an example:
{"log":"{\"message\":\"Something failed\",\"thread_name\":\"http-nio-8080-exec-1\",\"level\":\"WARN\",\"level_value\":30000,\"test-mdc\":\"test-mdc-value\",\"appname\":\"myWebservice\"}\n","stream":"stdout","time":"2017-11-30T14:44:19.514Z"}
I want to to unnest log of my app and move all fields on top level. Thanks to this answer Parsing nested JSON string in Logstash I managed to parse all jsons(although I don't understand anything from this config), but my app log is still not on the top level. Here is what I got:
{
"@version" => "1",
"host" => "4c11913cddf3",
"@timestamp" => 2017-12-01T07:50:16.562Z,
"message" => {
"test-mdc" => "test-mdc-value",
"appname" => "myWebservice",
"level" => "WARN",
"thread_name" => "http-nio-8080-exec-1",
"level_value" => 30000,
"message" => "Something failed"
}
}
My config:
input { stdin { } }
filter {
json {
source => "message"
target => "message"
}
json {
source => "[message][log]"
target => "[message]"
}
}
output { stdout { codec => rubydebug } }
Can I move all nested fields on top level overriding duplicates if there is one? Better without ruby, because I don't know it.