Parsing nested json with jq

Question

I am parsing a nested json to get specific values from the json response. The json response is as follows:

{ 
"custom_classes": 2,
"images": 
    [ 
      { 
        "classifiers": 
        [ 
          { 
            "classes": 
            [ 
              { 
                "class": "football",
                "score": 0.867376 
              } 
            ], 
            "classifier_id": "players_367677167",
            "name": "players" 
          } 
        ], 
        "image": "1496A400EDC351FD.jpg" 
      } 
    ], 
"images_processed": 1 
}

From the class images=>classifiers=>classes:"class" & "score" are the values that I want to save in a csv file. I have found how to save the result in a csv file. But I am unable to parse the images alone. I can get custom_classes and image_processed.

I am using jq-1.5.

The different commands I have tried :

curl "Some address"| jq '.["images"]'
curl "Some address"| jq '.[.images]'
curl "Some address"| jq '.[.images["image"]]'

Most of the times the error is about not being able to index the array images.

Any hints?

Template answer: Questions seeking debugging help ("why isn't this code working?") must include the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself. Questions without a clear problem statement are not useful to other readers. See: [How to create a Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve). — Oliver Baumann, Oct 26 '17 at 13:53
For extracting JSON embedded in a JSON string, see instead https://stackoverflow.com/questions/35154684/how-to-parse-a-json-string-with-jq-or-other-alternatives (TLDR: `jq '.field | fromjson'`) — tripleee, Jan 19 '22 at 12:47

score 13 · Accepted Answer · edited Oct 26 '17 at 16:13

13

I must say, I'm not terribly good at jq, so probably all those array iterations could be shorthanded somehow, but this yields the values you mentioned:

cat foo.json | jq ".[] | .images | .[] | .classifiers | .[] | .classes | .[] | .[]"

If you want the keys, too, just omit that last .[].`

Edit As @chepner pointed out in the comments, this can indeed be shortened to

cat foo.json | jq ".images[].classifiers[].classes[] | [.class, .score] | @csv "

edited Oct 26 '17 at 16:13

chepner

497,756
71
530
681

answered Oct 26 '17 at 14:26

Oliver Baumann

2,209
1
10
26

2

`.images[].classifiers[].classes[]` should be fine, at least in version 1.5 (which, honestly, has probably been out long enough that if people aren't using it, they should upgrade). The output of that filter could be piped through `[.class, .score] | @csv` to produce the desired CSV output. – chepner Oct 26 '17 at 14:32
@chepner: is it not possible that i access both the variables in the single line? `cat foo.json | jq '.images[].classifiers[].classes[]{.score,.class}' `?? I have tried differnt combinations. I am following the sample from a tutorial website. – Vini Oct 26 '17 at 14:53
@Vini, perhaps try this: `cat foo.json | jq '.[] | .images[].classifiers[].classes[] | {foo: .score, bar: .class}'`, you need to provide the object-keys to use in the newly constructed object. Or use `[.score, .class]` to make it an array. – Oliver Baumann Oct 26 '17 at 15:01
@OliverBaumann: i found it. I was trying all possibilities :) – Vini Oct 26 '17 at 15:02
@OliverBaumann : it did not work with the ` .[ ] ` in the initial place. The command from chepner simply works as well. – Vini Oct 26 '17 at 15:03

jq170727 · Answer 2 · 2017-10-26T17:39:00.363

Depending on the data this filter which uses Recursive Descent: .., objects and has may work:

.. | objects | select(has("class")) | [.class,.score] | @csv

Sample Run (assuming data in data.json)

$ jq -Mr '.. | objects | select(has("class")) | [.class,.score] | @csv' data.json
"football",0.867376

Try it online at jqplay.org

Here is another variation which uses paths and getpath

getpath( paths(has("class")?) ) | [.class,.score] | @csv

Try it online at jqplay.org

score 1 · Answer 3 · answered Oct 26 '17 at 15:08

1

jq solution to obtain a prepared csv record:

jq -r '.images[0].classifiers[0].classes[0] | [.class, .score] | @csv' input.json

The output:

"football",0.867376

answered Oct 26 '17 at 15:08

RomanPerekhrest

88,541
4
65
105

Parsing nested json with jq

3 Answers3

Linked