Logstash custom date log format match

Question

I have this log that print the date format that looks like this:

=          Build Stamp: 10:45:33 On Apr  4 2014           =

So i have run the filter on grok debugger but still clueless on how to remove the word On

grok {
patterns_dir => "./patterns"
match => { "message" => "%{F_TIMESTAMP:timestamp}" }
}

date {
match => [ "timestamp" , "HH:mm:ss MMM  d yyyy" , "HH:mm:ss MMM  dd yyyy" ]
locale => "en"
}

pattern file,

F_TIMESTAMP %{TIME} \On %{MONTH} +%{MONTHDAY} %{YEAR}

My current output for timestamp would be

10:45:33 On Apr 4 2014 on grok debugger.

Then how can i make it compatible/match with logstash @timestamp ?

score 8 · Accepted Answer · edited Apr 08 '17 at 11:08

8

You can extract each part of date time and combine in another field without On keyword.

You can achieve this following :

filter {
    grok {         
        match => { "message" => "%{F_TIMESTAMP}" }
    }
    mutate {
        add_field => { 
            "timestamp" => "%{time} %{month} %{monthday} %{year}"
        }
    }
    date {
        match => [ "timestamp" , "HH:mm:ss MMM d yyyy" , "HH:mm:ss MMM dd yyyy" ]
        locale => "en"
    }
    mutate {
        remove_field => [ "time" ,"month","monthday","year","timestamp"]
    }
}

F_TIMESTAMP %{TIME:time}\s*On\s*%{MONTH:month}\s*%{MONTHDAY:monthday}\s*%{YEAR:year}

Its working fine for me.

edited Apr 08 '17 at 11:08

Simon Forsberg

13,086
10
64
108

answered Sep 25 '14 at 11:52

Nirdesh Sharma

734
5
14

Thanks, this works for me but later i found out that the hour tag HH does not appears correctly in @timestamp" => "2014-04-04T02:45:33.000Z , as you can see the HH parts become 02 instead of 10. Then,i have removed the mutate remove_field tag to actually see what the hour timestamp capture the hour part shows 10 so it really puzzles me now. – moalt wisp Sep 25 '14 at 14:38
logstash changed the 10 into 2 because my location is +8 from UTC. So it will be different value if i run this at different location/country. I think i need to add timezone field for this. – moalt wisp Sep 25 '14 at 15:06
Yes..you need to convert it in your local time. Can be done by the ruby script as : `ruby { code => "event['@timestamp'] = event['@timestamp'].getlocal"}` – Nirdesh Sharma Sep 25 '14 at 17:48

Logstash custom date log format match

1 Answers1