0

I am using Logstash to parse a log file. A sample log line is shown below.

2011/08/10 09:51:34.450457,1.048908,tcp,213.200.244.217,47908, ->,147.32.84.59,6881,S_RA,0,0,4,244,124,flow=Background-Established-cmpgw-CVUT

I am using following filter in my confguration file.

 grok {
       match => ["message","%{DATESTAMP:timestamp},%{BASE16FLOAT:value},%{WORD:protocol},%{IP:ip},%{NUMBER:port},%{GREEDYDATA:direction},%{IP:ip2},%{NUMBER:port2},%{WORD:status},%{NUMBER:port3},%{NUMBER:port4},%{NUMBER:port5},%{NUMBER:port6},%{NUMBER:port7},%{WORD:flow}" ]
    }

It works well for error-free log lines. But when I have a line like below, it fails. Note that the second field is missing.

2011/08/10 09:51:34.450457,,tcp,213.200.244.217,47908, ->,147.32.84.59,6881,S_RA,0,0,4,244,124,flow=Background-Established-cmpgw-CVUT

I want to put a default value in there in my output Json object, if a value is missing. how can I do that?

user1097675
  • 33
  • 1
  • 2
  • 6

2 Answers2

0

Use (%{BASE16FLOAT:value})? for second field to make it optional - ie. regex ()? .

Even if the second field is null the grok will work.

So entire grok look like this:

%{DATESTAMP:timestamp},(%{BASE16FLOAT:value})?,%{WORD:protocol},%{IP:ip},%{NUMBER:port},%{GREEDYDATA:direction},%{IP:ip2},%{NUMBER:port2},%{WORD:status},%{NUMBER:port3},%{NUMBER:port4},%{NUMBER:port5},%{NUMBER:port6},%{NUMBER:port7},%{WORD:flow}
jijinp
  • 2,592
  • 1
  • 13
  • 15
0

Use it in your conf file. Now, if value field is empty it will omit it in response. 

input {
   stdin{
   }
}
filter {

grok {
       match => ["message","%{DATESTAMP:timestamp},%{DATA:value},%{WORD:protocol},%{IP:ip},%{NUMBER:port},%{GREEDYDATA:direction},%{IP:ip2},%{NUMBER:port2},%{WORD:status},%{NUMBER:port3},%{NUMBER:port4},%{NUMBER:port5},%{NUMBER:port6},%{NUMBER:port7},%{WORD:flow}" ]
    }

}
output {
  stdout {
        codec => rubydebug
  }
}
Mukrram Rahman
  • 426
  • 2
  • 14