How do I parse JSON log file with jq?

Question

I have two types of logs in a JSON log file and I want to parse and label each event with a tag using a jq filter. An example of each event below:

The goal is to label each event so that if message begins with a TR, .sourcetype=application_log, else if message begins with an IP, .sourcetype=access_log.

So far, I'm working with this: test.log jq -r '.[] | select(.log[12:14] == "TR") | .sourcetype = "application_log" | .sourcetype'

{
"log": "{\"message\":\"TR=failed to send order confirmation to \\\"someone@example.com\\\": rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \\\"transport: Error while dialing dial tcp 10.64.5.235:5000: i/o timeout\\\"\",\"severity\":\"warning\",\"timestamp\":\"2019-07-23T00:47:07.216693578Z\"}\n",
"stream": "stdout",
"time": "2019-07-23T00:47:07.222368843Z"
}

{
"log": "{\"message\":\"IP=failed to send order confirmation to \\\"someone@example.com\\\": rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \\\"transport: Error while dialing dial tcp 10.64.5.235:5000: i/o timeout\\\"\",\"severity\":\"warning\",\"timestamp\":\"2019-07-23T00:47:07.216693578Z\"}\n",
"stream": "stdout",
"time": "2019-07-23T00:47:07.222368843Z"
}

Possible duplicate of [Use jq to parse a JSON String](https://stackoverflow.com/questions/35154684/use-jq-to-parse-a-json-string) — Slai, Jul 23 '19 at 18:40

peak · Accepted Answer · 2019-07-23T21:48:18.127

If I understand the task correctly, a solution would be:

.log[12:14] as $code    
| if ($code == "TR") then .sourcetype = "application_log"
  elif ($code == "IP") then .sourcetype = "access_log"
  else .
  end

If you want the .log values as JSON objects so you can add the .sourcetype there, you would have to use fromjson on the original .log values, along the lines of:

.log |= fromjson
| .message[0:2] as $code    
| if ($code == "TR") then .log.sourcetype = "application_log"
  elif ($code == "IP") then .log.sourcetype = "access_log"
  else .
  end
| .log |= tostring . # is this line really needed?

How do I parse JSON log file with jq?

1 Answers1