1

I'm having problems using grok filter on logstash. I have this log:

83.149.9.216 - - [04/Jan/2015:05:13:42 +0000]

And I want to parse the IP and the date. I have the code below but I'm getting no matches.

^%{IPV4:req_id} - - \[(?<date>%{DAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} +0000)]

What am I doing wrong? Thanks!

SLFl
  • 193
  • 1
  • 2
  • 13

1 Answers1

3

You should change %{DAY} (=day of the week name) to %{MONTHDAY} (to match the numbers) and escape the + to match it as a literal + char:

^%{IPV4:req_id} - - \[(?<date>%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} \+0000)]
                              ^^^^^^^^^^^                                              ^

As suggested by Calvin Taylor, you may further enhance the pattern to match any ISO8601 time zone with %{ISO8601_TIMEZONE} instead of \+0000:

^%{IPV4:req_id} - - \[(?<date>%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{ISO8601_TIMEZONE})]
                                                                                       ^^^^^^^^^^^^^^^^^^^

See Grok patterns:

MONTHDAY (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])
DAY (?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)
Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
  • Thanks! Just one more question. I have this: `^%{IPV4:req_id} - - \[(?%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{ISO8601_TIMEZONE})] "(?%{WORD}) (?%{UNIXPATH}) (?[^"]*)" %{NUMBER:number} %{NUMBER:number1} "(?[^"]*)" "(?[^/]*)/(?[^\s]*) \((?[^;]*); (?[^)]*)\) (?.*)$` it is working well in grok debugger but when I run in logstash I have a timeout error. Do you know why? – SLFl Nov 08 '17 at 11:32
  • @SLFlor It makes no sense: the pattern seems to be written well (linear pattern matching is ensured). Is there any specific error shown? Just timeout? – Wiktor Stribiżew Nov 08 '17 at 11:38
  • Just "Timeout executing grok". But if I wait a little bit it works. Strange thing – SLFl Nov 08 '17 at 11:47
  • @SLFlor I checked the Grok patterns and the `%{NUMBER}` is a bit "expensive". It may be the culprit here. Try replacing `%{NUMBER:number}` with `(?[+-]?\d*\.?\d+)`. Or even `(?[+-]?\d+(?:\.\d+)?)`. Same with `number1`. Or `(?[+-]?(?>\d+(?:\.\d+)?|\.\d+))` - just make sure you match the right number format. – Wiktor Stribiżew Nov 08 '17 at 11:53