3

I want to use Grok Pattern for filtering out this

172.20.20.88 - - [10/Nov/2018:23:49:31 +0700] "GET /id/profile.pl?user=285&device=Bg3tlX HTTP/1.1" 502 852 "-" "Go-http-client/2.0" "0.009"

I am using COMMONAPACHELOG

%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)

I have tried %{URIPATH:request} and %{URIPARAM:request}. The result of request is still /id/profile.pl?user=285&device=Bg3tlX. My expectation is /id/profile.pl.

My reference is https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns

Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
Merces Koba
  • 33
  • 4

1 Answers1

0

Your %{NOTSPACE:request} matches any 1 or more non-whitespace chars before HTTP/1.1" 502 85... as NOTSPACE pattern is \S+. So, it matches the whole /id/profile.pl?user=285&device=Bg3tlX substring.

You cannot use just URIPATH or URIPARAM, because you still need to match the rest of the input. You have to use both, but make URIPARAM optional after URIPATH by enclosing it within an optional non-capturing group, (?:...)?.

So, replace %{NOTSPACE:request} with

%{URIPATH:request}(?:%{URIPARAM:requestparam})?
                  ^^^                        ^^

Demo at https://grokdebug.herokuapp.com/:

enter image description here

Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
  • Thank you so much... thank you thank you... I got stuck for 8 hours... almost give up... using regex manually one-by-one crossing in my mind... but you helped me solve it... – Merces Koba Nov 11 '18 at 05:57