How can I use Jq to help me make "individual" json files with specific values?

Question

I want to use jq to help me make a file for every key, with the 3 specific values pertaining to that key, formatted in json.

I know how to get my keys, and I know how to get my values. But how can I do all this in one command so that I have the info I want, in different files, per key with the keys respective values?

Here is how I get keys:

cat $DCAR_API | jq .vendors[].products[].images | jq -r 'keys[]'

Here is the info I want to be in every json file:

cat "$DCAR_API" \
| jq -r '.vendors[]|.products[]|.images[].tags[].pgpSignature' 

cat "$DCAR_API" \
| jq -r '.vendors[]|.products[]|.images[].tags[].imageManifest'

cat "$DCAR_API" \
| jq -r '.vendors[]|.products[]|.images[].tags[].image'

Can you show us an actual JSON with the above information and an exact output needed out of it? — Inian, Feb 17 '20 at 20:13
`jq` *doesn't* create files. It doesn't have the facilities to do so at all. — Charles Duffy, Feb 17 '20 at 20:14
Hmm. Arguably, this is an (overcomplicated) duplicate of [Using `jq`, how can I split a JSON stream of objects into separate files based on the values of an object property?](https://stackoverflow.com/questions/54725080/using-jq-how-can-i-split-a-json-stream-of-objects-into-separate-files-based-on). Any questions that are related not to the actual split operation but on how to operate on the OP's data should be split out and asked separately. — Charles Duffy, Feb 17 '20 at 20:40
...and for that matter, the biggest problem with the OP's existing data-wrangling code here (unrelated to the split) is one with a *separate* duplicate answering it, [jq query returning too many records / unwanted permutations](https://stackoverflow.com/questions/60105691/jq-query-returning-too-many-records-unwanted-permutations). — Charles Duffy, Feb 17 '20 at 20:42

score 2 · Accepted Answer · answered Feb 17 '20 at 20:36

Since you haven't provided sample data, I've had to make my own:

json='
{ "vendors":
  { "vendorA":
    { "products":
      { "productA":
        { "images":
          { "imageA":
            { "tags":
              { "2019-01-01":
                { "pgpSignature":  "productA-imageA-20190101-sig",
                  "imageManifest": "productA-imageA-20190101-manifest",
                  "image":         "productA-imageA-20190101-image"
                }
              }
            }
          }
        }
      }
    }
  }
}
'

buildJsonStream() {
  jq -j '
    .vendors[]|.products[]|.images | to_entries[] |
    .key as $key |
    .value.tags[] |
    ($key, "\u0000",
     ({ pgpSignature, imageManifest, image } | tojson), "\n", "\u0000"
    )' "$@"
}

declare -A alreadyCreated=( )
while IFS= read -r -d '' key && IFS= read -r -d '' content; do
  if ! [[ ${alreadyCreated[$key]} ]]; then
    printf '%s' "$content" >"$key.json"
    alreadyCreated[$key]=1
  else
    printf '%s' "$content" >>"$key.json"
  fi
done < <(buildJsonStream <<<"$json")

...will write, to imageA.json, the text:

{"pgpSignature":"productA-imageA-20190101-sig","imageManifest":"productA-imageA-20190101-manifest","image":"productA-imageA-20190101-image"}

The important thing to note here is that we're having jq write alternating, NUL-separated key/content pairs, and then having bash be responsible for actually writing to the (key-name-derived) files.

The effect of the call to `tojson` is to squish the JSON onto one line. Why not omit it? — peak, Feb 17 '20 at 23:15
Unless there are restrictions on the OP's input format they aren't telling us about, there can be multiple images per key and multiple tags per image, and thus many signature/manifest/image tuples per output file. Using JSONL makes sense. — Charles Duffy, Feb 17 '20 at 23:21

How can I use Jq to help me make "individual" json files with specific values?

1 Answers1