How to parse JSON in shell script?

Question

I run the curl command $(curl -i -o - --silent -X GET --cert "${CERT}" --key "${KEY}" "$some_url") and save the response in the variable response. ${response} is as shown below

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 34
Connection: keep-alive
Keep-Alive: timeout=5
X-XSS-Protection: 1; 

{"status":"running","details":"0"}

I want to parse the JSON {"status":"running","details":"0"} and assign 'running' and 'details' to two different variables where I can print status and details both. Also if the status is equal to error, the script should exit. I am doing the following to achieve the task -

status1=$(echo "${response}" | awk '/^{.*}$/' | jq -r '.status')
details1=$(echo "${response}" | awk '/^{.*}$/' | jq -r '.details')
echo "Status: ${status1}"
echo "Details: ${details1}"
if [[ $status1 == 'error' ]]; then
    exit 1
fi

Instead of parsing the JSON twice, I want to do it only once. Hence I want to combine the following lines but still assign the status and details to two separate variables -

status1=$(echo "${response}" | awk '/^{.*}$/' | jq -r '.status')
details1=$(echo "${response}" | awk '/^{.*}$/' | jq -r '.details')

What does your `curl` command look like? If you're throwing away the headers anywaym, you can keep them out of `$response` in the first place. — Benjamin W., Nov 27 '18 at 22:08

Charles Duffy · Accepted Answer · 2018-11-27T22:46:16.297

First, stop using the -i argument to curl. That takes away the need for awk (or any other pruning of the header after-the-fact).

Second:

{
  IFS= read -r -d '' status1
  IFS= read -r -d '' details1
} < <(jq -r '.status + "\u0000" + .details + "\u0000"' <<<"$response")

The advantage of using a NUL as a delimiter is that it's the sole character that can't be present in the value of a C-style string (which is how shell variables' values are stored).

score 1 · Answer 2 · answered Nov 27 '18 at 22:20

1

You can use a construction like:

read status1 details1 < <(jq -r '.status + " " + .details' <<< "${response}")

You use read to assign the different inputs to two variables (or an array, if you want), and use jq to print the data you need separated by whitespace.

answered Nov 27 '18 at 22:20

Poshi

5,332
3
15
32

2

If you want to allow for arbitrary strings instead of relying on whitespace, you could use a nul byte as per https://stackoverflow.com/questions/49321015/storing-jq-null-delimited-output-in-bash-array – Benjamin W. Nov 27 '18 at 22:26

score 1 · Answer 3 · answered Nov 27 '18 at 22:34

As Benjamin already suggested, only retrieving the json is a better way to go. Poshi's solution is solid.

However, if you're looking for the most compact to do this, no need to save the response as a variable if the only thing your're going to do with it is extract other variables from it on a one time basis. Just pipe curl directly into:

curl "whatever" | jq -r '[.status, .details] |@tsv'

or

curl "whatever" | jq -r '[.status, .details] |join("\t")'

and you'll get your values fielded for you.

How to parse JSON in shell script?

3 Answers3