Parse JSON and JSON values with jq

Question

I have an API that returns JSON - big blocks of it. Some of the key value pairs have more blocks of JSON as the value associated with a key. jq does a great job of parsing the main JSON levels. But I can't find a way to get it to 'recurse' into the values associated with the keys and pretty print them as well.

Here is the start of one of the JSON returns. Note it is only a small percent of the full return:

    {
  "code": 200,
  "status": "OK",
  "data": {
    "PlayFabId": "xxxxxxx",
    "InfoResultPayload": {
      "AccountInfo": {
        "PlayFabId": "xxxxxxxx",
        "Created": "2018-03-22T19:23:29.018Z",
        "TitleInfo": {
          "Origination": "IOS",
          "Created": "2018-03-22T19:23:29.033Z",
          "LastLogin": "2018-03-22T19:23:29.033Z",
          "FirstLogin": "2018-03-22T19:23:29.033Z",
          "isBanned": false
        },
        "PrivateInfo": {},
        "IosDeviceInfo": {
          "IosDeviceId": "xxxxxxxxx"
        }
      },
      "UserVirtualCurrency": {
        "GT": 10,
        "MB": 70
      },
      "UserVirtualCurrencyRechargeTimes": {},
      "UserData": {},
      "UserDataVersion": 15,
      "UserReadOnlyData": {
        "DataVersion": {
          "Value": "6",
          "LastUpdated": "2018-03-22T19:48:59.543Z",
          "Permission": "Public"
        },
        "achievements": {
          "Value": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50,\"achievements\":[{\"id\":2,\"name\":\"Correct Round 4\",\"description\":\"Round 4 answered correctly\",\"maxValue\":10,\"increment\":1,\"currentValue\":3,\"valueUnit\":\"unit\",\"awardOnIncrement\":true,\"marbles\":10,\"image\":\"https://www.jamandcandy.com/kissinkuzzins/achievements/icons/sphinx\",\"SuccessKey\":[\"0_3_4_0\",\"0_5_4_0\",\"0_6_4_0\",\"0_7_4_0\",\"0_8_4_0\",\"0_9_4_0\",\"0_10_4_0\"],\"event\":\"Player_answered_round\",\"achieved\":false},{\"id\":0,\"name\":\"Complete

This was parsed using jq but as you can see when you get to the

"achievements": { "Vales": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50,\

lq does no further parse the value at is also JSON.

Is there a filter I am missing to get it to parse the values as well as the higher level structure?

Possible duplicate of [Use jq to parse a JSON String](https://stackoverflow.com/questions/35154684/use-jq-to-parse-a-json-string) — Jeff Mercado, Mar 22 '18 at 21:02
Please ensure your samples are complete, e.g. the main example should be valid JSON, and in the snippet, the value of "achievements" should be valid JSON. — peak, Mar 22 '18 at 21:43
`Value` does not map to more structured JSON. It maps to a single string, that happens to contain JSON escaped and stored as a string. Any valid JSON parser would not parse that value further; the escapes exist explicitly to prevent the parser from attempting to do so, at least until it's invoked explicitly on that string. If any of your code is generating this JSON, it's doing so wrongly. — jpmc26, Mar 22 '18 at 22:17

peak · Answer 1 · 2018-03-22T21:39:31.530

Is there a filter I am missing ...?

The filter you'll need is fromjson, but it should only be applied to the stringified JSON; consider therefore using |= as illustrated using your fragment:

echo '{"achievements": { "Vales": "[{\"id\":0,\"gamePack\":\"GAME.PACK.0.KK\",\"marblesAmount\":50}]"}}' | 
  jq '.achievements.Vales |= fromjson'

{
  "achievements": {
    "Vales": [
      {
        "id": 0,
        "gamePack": "GAME.PACK.0.KK",
        "marblesAmount": 50
      }
    ]
  }
}

recursively/1

If you want to apply fromjson recursively wherever possible, then recursively is your friend:

def recursively(f):
  . as $in
  | if type == "object" then
      reduce keys[] as $key
        ( {}; . + { ($key):  ($in[$key]  | recursively(f) )} )
  elif type == "array" then map( recursively(f) )
  else try (f as $f | if $f == . then . else ($f | recursively(f)) end) catch $in
  end;

This would be applied as follows:

recursively(fromjson)

Example

{a: ({b: "xyzzy"}) | tojson} | tojson 
| recursively(fromjson)

yields:

{
  "a": {
    "b": "xyzzy"
  }
}

Super this recursing into the values is what I was looking to do. I will see what I can put together using these ideas. I realize that it was a string as far as the enclosing JSON. I still wanted a way to recurse into it and parse it too - even though it is escaped. This seems like the right direction to try. Thanks again! — Dale Strickler, Mar 23 '18 at 14:53
Thank you! This answer is amazing. Particularly the use of your `recursively` function. Previously I was using `[to_entries[] | .value |= fromjson] | from_entries` but that only works when all values are json strings. It was a mess trying to genericly select the valid json string values and then merge them with the non-json string key/values. Using `recursively` just solves this quite elegantly! Many thanks — Chris, Sep 08 '22 at 00:01

Parse JSON and JSON values with jq

1 Answers1

recursively/1

Example