JQ: filter nested array of objects in only include selected fields

Question

Starting with the following, I just want the .title, .url, .author.login, .reviews.author.login, .reviews.state and .mergedBy.login fields? I'm getting close with map({ title: .title, URL: .url, author: .author.login, reviews: .reviews, "merged by": .mergedBy.login}) but how do I also limit the fields in the reviews array to author.login and state?

[
  {
    "author": {
      "id": "MDQ6VXNlcjYGJOYzNTIw",
      "is_bot": false,
      "login": "luke123",
      "name": "Luke"
    },
    "mergedAt": "2023-01-12T20:29:41Z",
    "mergedBy": {
      "id": "MDQ6VXNlcjYGJOYzNTIw",
      "is_bot": false,
      "login": "luke123",
      "name": "Luke"
    },
    "reviews": [
      {
        "id": "PRR_kwDODrt9D55KSvlh",
        "author": {
          "login": "han123"
        },
        "authorAssociation": "CONTRIBUTOR",
        "body": "",
        "submittedAt": "2023-01-12T20:26:50Z",
        "includesCreatedEdit": false,
        "reactionGroups": [],
        "state": "APPROVED",
        "commit": {
          "oid": "ff541b49775eab1ffc63www850dc405ecabc045f"
        }
      },
      {
        "id": "PRR_kwDODrt7OM8WHwat",
        "author": {
          "login": "leia"
        },
        "authorAssociation": "CONTRIBUTOR",
        "body": "",
        "submittedAt": "2023-01-12T20:29:15Z",
        "includesCreatedEdit": false,
        "reactionGroups": [],
        "state": "APPROVED",
        "commit": {
          "oid": "dd541b49775eab1ffc63bbb850dc405edhwc045c"
        }
      }
    ],
    "title": "Jedi Training",
    "url": "https://github.com/rebels/training/pull/222"
  },
  ...
]

Please follow the [mcve] guidelines, or at least clarify whether you want the structure to be preserved. — peak, Mar 01 '23 at 15:06
Thanks for everyone's input! My command became alot bigger when trying to convert to csv but I got there. — David Williams, Mar 02 '23 at 15:01

score 2 · Answer 1 · answered Mar 01 '23 at 15:00

2

One of many possibilities for preserving the structure:

…, reviews: (.reviews | map({author,state} | .author|= {login})) …

The technique of writing {foo} as an abbreviation for {"foo": .foo} comes in handy.

answered Mar 01 '23 at 15:00

peak

105,803
17
152
177

score 1 · Answer 2 · answered Mar 01 '23 at 14:57

Use the same technique with map({…}) as you did on the outer object:

map({
  title,
  URL: .url,
  author: .author.login,
  reviews: (.reviews | map({
    author: .author.login,
    state
  })),
  "merged by": .mergedBy.login
})

[
  {
    "title": "Jedi Training",
    "URL": "https://github.com/rebels/training/pull/222",
    "author": "luke123",
    "reviews": [
      {
        "author": "han123",
        "state": "APPROVED"
      },
      {
        "author": "leia",
        "state": "APPROVED"
      }
    ],
    "merged by": "luke123"
  }
]

Note: To just replicate a field, e.g. {title: .title}, you can shorten it to just {title}.

knittl · Answer 3 · 2023-03-01T15:19:08.563

You can easily nest map calls. Just make sure that you group value expressions with a pipe:

map({
    title,
    URL: .url,
    author: .author.login,
    reviews: (.reviews | map({"reviewed by": .author.login, state})),
    "merged by": .mergedBy.login
})

Output:

[
  {
    "title": "Jedi Training",
    "URL": "https://github.com/rebels/training/pull/222",
    "author": "luke123",
    "reviews": [
      {
        "reviewed by": "han123",
        "state": "APPROVED"
      },
      {
        "reviewed by": "leia",
        "state": "APPROVED"
      }
    ],
    "merged by": "luke123"
  }
]

score 0 · Answer 4 · answered Mar 01 '23 at 14:57

By using .reviews you say: Take the complete reviews data and reuse it.

Similar to what you are doing for the complete list you could also map that array and only extract the required data for each entry:

jq 'map({ title: .title, URL: .url, author: .author.login, reviews: .reviews | map({ author: .author.login, state: .state }), "merged by": .mergedBy.login})'

The pipe operator | allows you to apply a filter to the result of the .reviews expression.

protob · Answer 5 · 2023-03-01T15:16:33.063

0

Try this to extract the required data:

jq '[.[] |
    {
      title: .title,
      URL: .url,
      author: .author.login,
      reviews: [.reviews[] | {
        "author": .author.login,
        "state": .state
      }],
      "merged by": .mergedBy.login
    }
  ]' data.json

edited Mar 01 '23 at 15:16

answered Mar 01 '23 at 15:11

protob

3,317
1
8
19

score 0 · Answer 6 · answered Mar 02 '23 at 11:08

Starting off with your own solution, restricting all nests within "reviews" to two keys - "author" and "state" - and finally, setting "author" to .author.login

map({ 
  title: .title, 
  URL: .url, 
  author: .author.login, 
  reviews: [(.reviews[] | with_entries(select(.key == "author" or .key == "state")) | .author=.author.login)], 
  "merged by": .mergedBy.login
})

Result:

[
  {
    "title": "Jedi Training",
    "URL": "https://github.com/rebels/training/pull/222",
    "author": "luke123",
    "reviews": [
      {
        "author": "han123",
        "state": "APPROVED"
      },
      {
        "author": "leia",
        "state": "APPROVED"
      }
    ],
    "merged by": "luke123"
  }
]

JQ: filter nested array of objects in only include selected fields

6 Answers6