1

I'm currently working through an issue, and can't seem to figure this one out. Here's some data so you know what I'm talking about below:

foo.json

{
    "Schedule": [
        {
            "deviceId": 123,
            "reservationId": 123456,
            "username": "jdoe"
        },
        {
            "deviceId": 456,
            "reservationId": 589114,
            "username": "jsmith"
        }
    ],
    "serverTime": 1522863125.019958
}

bar.json

[
    {
        "a": {
            "b": "10.0.0.1",
            "c": "hostname1"
        },
        "deviceId": 123
    },
    {
        "a": {
            "b": "10.0.0.2",
            "c": "hostname2"
        },
        "deviceId": 456
    }
]

foobar.json

{
    "Schedule": [
        {
            "deviceId": 123,
            "reservationId": 123456,
            "username": "jdoe",
            "a": {
                "b": "10.0.0.1",
                "c": "hostname1"
            }
        }
        },
        {
            "deviceId": 456,
            "reservationId": 789101,
            "username": "jsmith",
            "a": {
                "b": "10.0.0.2",
                "c": "hostname2"
            }
        }
    ],
    "serverTime": 1522863125.019958
}

I'm trying to use jq to do this, and had some help from this post: https://github.com/stedolan/jq/issues/1090 The goal is to be able to combine JSON, using some key as a common point between the documents. The data may be nested any amount of levels.. In this case foo.json has nested data only two levels deep, but needs to be combined with data nested 1 level deep.

Any and all suggestions would be super helpful. I'm also happy to clarify and answer questions if needed. Thank you!

peak
  • 105,803
  • 17
  • 152
  • 177
Daniel
  • 13
  • 3

1 Answers1

3

With foobar.jq as follows:

def dict(f):
  reduce .[] as $o ({}; .[$o | f | tostring] = $o ) ;

($bar | dict(.deviceId)) as $dict
| .Schedule |= map(. + ($dict[.deviceId|tostring] ))

the invocation:

jq -f foobar.jq --argfile bar bar.json foo.json

yields the output shown below.

Notice that the referents in the dictionary contain the full object (including the key/value pair for "deviceId"), but it's not necessary to del(.deviceId) because of the way + is defined in jq.

Output

{
  "Schedule": [
    {
      "deviceId": 123,
      "reservationId": 123456,
      "username": "jdoe",
      "a": {
        "b": "10.0.0.1",
        "c": "hostname1"
      }
    },
    {
      "deviceId": 456,
      "reservationId": 589114,
      "username": "jsmith",
      "a": {
        "b": "10.0.0.2",
        "c": "hostname2"
      }
    }
  ],
  "serverTime": 1522863125.019958
}
peak
  • 105,803
  • 17
  • 152
  • 177
  • This worked splendidly! Thank you very much. I do have one additional ask. How would you recommend that I invoke jq in this way from python 3? I'm fine with using the subprocess module, but I'm open to doing things a better way if one exists. – Daniel May 12 '18 at 22:09
  • From the jq FAQ: : What language bindings are available for Python? A: pip install jq # For details, see https://pypi.python.org/pypi/jq pip install pyjq # For details, see https://pypi.python.org/pypi/pyjq – peak May 12 '18 at 22:42