Bash script to parse log file

Question

I am trying to parse a log file to extract userIds. Below is how each log is setup in catalina.out. I want to extract just the userId. How would I do that. I tried awk and grep but it returns the entire block instead of just userId. Also, I want to get every unique userId once, because the same user could log in multiple times obviously so I just want the file I write it to to just have it once. Could you guys please help me get an idea where to start? Thanks!

Here are the commands I've tried:

awk '/userId/' catalina.out

grep "userId" catalina.out

When I do this instead of returning back the userId, it returns back the entire block (as seen below)

Log format:

03:44:04.373 [127.0.0.1-8009-exec-178] 
INFO  c.c.c.x.x.w.f.AuthenticationFilter - cachObj
{"guid":"guid","userId":"userId","isPrimary":false,"accessToken":"accessToken"}

Looks like JSON output, use a proper parser for it – Inian Jan 09 '17 at 15:03 — Inian, Jan 09 '17 at 15:03
Is your log sample 1 line or multiple? – Zlemini Jan 09 '17 at 21:42 — Zlemini, Jan 09 '17 at 21:42

score 0 · Answer 1 · edited Jan 10 '17 at 13:48

0

I am assuming your userids will be found like "userId":"test_chumma" etc etc, if this is the case then following may help you in same.

awk -F'[":,]' '/userId/{print $11}'  Input_file

Off course if you have more requirements then kindly show us more sample output with full conditions.

edited Jan 10 '17 at 13:48

Matt

74,352
26
153
180

answered Jan 09 '17 at 14:41

RavinderSingh13

130,504
14
57
93

I tried using your command with my input file name and I don't get back any results. – redsox2002 Jan 09 '17 at 15:20
@redsox2002: Kindly post more sample Input_file into post and let us know then. – RavinderSingh13 Jan 09 '17 at 15:24

JNevill · Answer 2 · 2017-01-09T16:09:42.057

0

Another example with awk:

awk -F"[:,]" '$1 ~ /^{/ {gsub("\"","",$4); print $4}' inFile.log

Here we split the records by a colon : or a comma , using the awk command's F flag -F[:,], Then if the first field in the record $1 starts with a bracket /^{/ (where we use the regex operator ~ for the condition) then we swap out the 4th field's double quotes with nothing gsub("\"","",$4) and print the result print $4

$ cat test
03:44:04.373 [127.0.0.1-8009-exec-178]
INFO  c.c.c.x.x.w.f.AuthenticationFilter - cachObj
{"guid":"guid","userId":"aUserId","isPrimary":false,"accessToken":"accessToken"}
$ awk -F"[:,]" '$1 ~ /^{/ {gsub("\"","",$4); print $4}' test
aUserId

edited Jan 09 '17 at 16:09

answered Jan 09 '17 at 14:44

JNevill

46,980
4
38
63

I unfortunately don't get back any results using this command – redsox2002 Jan 09 '17 at 15:20
I used your three line example log file from above. I will post the exact results of running this on my box. Perhaps your log file is formatted differently then your example? – JNevill Jan 09 '17 at 16:09

score 0 · Answer 3 · answered Jan 10 '17 at 10:02

0

Here is another solution combining awk and cut from unix:

awk '{split($0,a,":"); print a[2]}' catalina.out | cut -f2 -d","
"userId"

But this will also work only for the example you posted.

answered Jan 10 '17 at 10:02

JFS31

518
5
13

score 0 · Answer 4 · answered Jan 10 '17 at 11:12

0

awk -F\" '{print $6}' file 

userId

answered Jan 10 '17 at 11:12

Claes Wikner

1,457
1
9
8

Bash script to parse log file

4 Answers4