grok - how do you find a quoted string

Question

I am trying to grab the output from an nginx log file and send it to logstash.

10.1.10.20 - bob [14/Feb/2014:18:57:05 +0000] “POST /main/foo.git/git-upload-pack HTTP/1.1” 200 3653189 “-” “git/1.8.3.4 (Apple Git–47)”

Grock is able to find the first 3 words fine

10.1.10.20 - bob [14/Feb/2014:18:57:05 +0000]

%{IPV4:user_ip} - %{USERNAME:user_name} \[%{HTTPDATE:time_local}\]

Grok is able to find the 3rd and 4th words fine

[14/Feb/2014:18:57:05 +0000] “POST /main/foo.git/git-upload-pack HTTP/1.1”

\[%{HTTPDATE:time_local}\] %{QUOTEDSTRING:request}

However when I combine them, and try to find all 4, grok says there are no results (using http://grokdebug.herokuapp.com/ for testing)

10.1.10.20 - bob [14/Feb/2014:18:57:05 +0000] “POST /main/foo.git/git-upload-pack HTTP/1.1” 

%{IPV4:user_ip} - %{USERNAME:user_name} \[%{HTTPDATE:time_local}\]  %{QUOTEDSTRING:request}
#not found

Anyone know how to get the quoted string in the above example?

I'm brand new to grok, so perhaps I'm not approaching this correctly.

Update

Interestingly if I use the following log line and then manually type in the url it does work

 bob 14/Feb/2014:18:57:05 +0000 "herp"
 #Once herp works, replace herp, with POST
 bob 14/Feb/2014:18:57:05 +0000 "POST"
 #Once POST works, keep expounding until the whole thing is in place
 autobuild 14/Feb/2014:18:57:05 +0000 "POST /main/builder.git/git-upload-pack HTTP/1.1"

Hi, did you manage to send the data to elasticsearch with the right type format (eg field time_local recognised as a date) ? — Alexandre Mélard, Jul 11 '14 at 13:53

score 3 · Answer 1 · answered Dec 01 '14 at 02:30

3

"POST /main/builder.git/git-upload-pack HTTP/1.1" in pattern

"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}"

answered Dec 01 '14 at 02:30

HDT

2,017
20
32

spuder · Accepted Answer · 2014-02-15T01:37:47.870

0

The process of posting to stack overflow identified the problem.

If you look carefully, the double quotes are parsed differently

"POST

vs

“POST

Manually typing in the double quote fixes the problem

edited Feb 15 '14 at 01:37

answered Feb 15 '14 at 00:40

spuder

17,437
19
87
153

Pablo Garcia · Answer 3 · 2015-09-10T20:42:16.583

0

Also you can use this expression for the cases where the log changes:

"%{WORD:verb}(?:| %{URIPATHPARAM:request})(?:| HTTP/%{NUMBER:httpversion})"

it matches with:

"POST /main/builder.git/git-upload-pack HTTP/1.1"

or

"POST /main/builder.git/git-upload-pack"

or

"POST"

try it.. ;)

edited Sep 10 '15 at 20:42

answered Jul 21 '15 at 16:05

Pablo Garcia

138
10

grok - how do you find a quoted string

3 Answers3