I stuck this issue for last 2 days. I am using using logstash to receive logs from the forwarder.I am using multiline filter to match the logs and parse the logs using grok pattern.All are working fine. Except the last line of log.
multiline {
patterns_dir => "/patterns"
pattern => "^\[%{OBIEE_DATESTAMP}\]"
negate => true
what => "previous"
enable_flush => true
stream_identity => "%{host}.%{type}"
}
if [type] == "nqcluster" {
grok {
patterns_dir => "/patterns"
match => [ "message", "%{OBIEE_CC_LOG}" ]
}
}
After i have used enable_flush option, It takes the last line of log as well. But it produces duplicate tag values as follows.
"timestamp" => [
[0] "2015-02-21T12:10:39.000+05:30",
[1] "2015-02-21T12:10:39.000+05:30"
],
"AppName" => [
[0] "OracleBIClusterControllerComponent",
[1] "OracleBIClusterControllerComponent"
],
Please help me to resolve this.