java regex for nginx logs

Question

I wish to transform my nginx logs using a reqular expression script as follows:

original log:

07.21.99.178 - - [01/Jun/2012:12:06:23 +0530] "GET /api?playSessionId=live_21_bc206d95-113f-4b49-989b-7dff77af51c410.190.217.2111338532565422 HTTP/1.1" 200 71 "-" "Jakarta Commons-HttpClient/3.1"

I want the playSessionId as the output for which I used the following script:

#!/usr/bin/env ruby

mon={"Jan" => '01',"Feb" => '02',"Mar" => '03',"Apr" => '04',"May" => '05',"Jun" =>        '06',"Jul" => '07',"Aug" => '08',"Sep" => '09',"Oct" => '10',"Nov" => '11',"Dec" => '12'}

STDIN.each_line do |line|
if line =~ /([\d+|\.]+) (\d+)\/(\w+)\/(\d+):(\d+):\d+:\d+ \+\d+] "GET \/api\?playSessionId=(^&*)/
d = "#{$3}-#{mon$2}-#{$1}"
h = $4
pid = $5
puts "#{d}\t#{h}\t#{pid}"
end
 end

But this doesn't seem to work :( can someone tell me the java regex for this so that i can sue rlike on hive?

If I understand correctly, you want this string `live_21_bc206d95-113f-4b49-989b-7dff77af51c410.190.217.2111338532565422` — m0skit0, Jun 14 '12 at 09:39

score 0 · Answer 1 · answered Jun 14 '12 at 09:40

0

I think you want this:

/([\d+|\.]+) (\d+)\/(\w+)\/(\d+):(\d+):\d+:\d+ \+\d+] "GET \/api\?playSessionId=([^ &]+)/

You might want to use standard unix tools for this job instead (grep + sed):

grep 'playSessionId=' foo.log | sed 's/^.*playSessionId=\([^ ]*\).*$/\1/'

answered Jun 14 '12 at 09:40

troelskn

115,121
27
131
155

I am using it for converting s3 raw logs to processed logs for hive on EMR – princess of persia Jun 14 '12 at 09:43

score 0 · Answer 2 · answered Jun 14 '12 at 09:43

0

I think youre regex is way too complicated. This will do the job:

playSessionId=(.*)\s

answered Jun 14 '12 at 09:43

m0skit0

25,268
11
79
127

I don't understand the question. This is a regular expression. – m0skit0 Jun 14 '12 at 09:48
I am putting the script on s3 and using it to transform my hive table how do i use this to do the job ? – princess of persia Jun 14 '12 at 09:48

java regex for nginx logs

2 Answers2