0

Sample input1.json


[{
    "organizationId": "org1",
    "test1": "UP",
    "key1": "value1"
}, 
{
    "organizationId": "org2",
    "test1": "UP",
    "key1": "value3"
}]

Sample input2.json


[{
    "organizationId": "org1",
    "test2": "DOWN",
    "key1": "value4"
}, 
{
    "organizationId": "org3",
    "test2": "DOWN",
    "key1": "value5"
}]

Expected output.json


[{
        "organizationId": "org1",
        "test1": "UP",
        "key1": "value4",
        "test2": "DOWN"
    },
    {
        "organizationId": "org2",
        "test1": "UP",
        "key1": "value3"
    },
    {
        "organizationId": "org3",
        "test2": "DOWN",
        "key1": "value5"
    }
]

The input is an array object of two files. My objective is to merge two objects if they have same value and leave other objects intact. I partially achieved this by grouping

jq -s '[ .[0] + .[1] | group_by(.organizationId)[] | select(length > 1) |add ]' ìnput1.json input2.json

Group objects by organizationId. In both the inputs, "organizationId": "org1" is available so it can be grouped. Now the problem I'm facing is I'm loosing other objects "organizationId": "org2" from input1.json and "organizationId": "org3" from input2.json which doesn't exist in each others file.

The basic principle of grouping is achieved but I do need to preserve any other objects from both the files even if there is no match. Should we use group_by if we want to preserve other objects? If not, how can I achieve the expected output using jq?

Naveen K Reddy
  • 429
  • 5
  • 13
  • but your current approach preserves all other ungrouped objects and gives you the expected result. What is the issue? – RomanPerekhrest Mar 31 '18 at 12:39
  • @RomanPerekhrest Thanks for spotting! I was trying different options and missed it! I updated the post with the one I tried and given answer also. – Naveen K Reddy Mar 31 '18 at 15:28

1 Answers1

0

For wider audience, solution to get all objects (grouped & ungrouped), use below

jq -s '[ .[0] + .[1] | group_by(.organizationId)[] |add ]' ìnput1.json input2.json

If only grouped objects, then use below.

jq -s '[ .[0] + .[1] | group_by(.organizationId)[] | select(length > 1) | add ]' ìnput1.json input2.json

I have to say jq is so powerful! :)

Naveen K Reddy
  • 429
  • 5
  • 13
  • (1) `.[0]+.[1]` can in this case be simplified to just: `add`; (2) group_by is a bit tricky in that the sorting algorithm it uses in version 1.5 is NOT guaranteed to be stable. A "stable" version was introduced on January 18, 2016. If you are stuck with an older version, then you could use GROUPS_BY as defined at https://stackoverflow.com/questions/48321235 – peak Apr 01 '18 at 19:41
  • @peak Thanks for letting us know. We are using `1.5-1-a5b5cbe`. I hope this is stable? Am I right? – Naveen K Reddy Apr 03 '18 at 13:10
  • a5b5cbe dates from Aug 18, 2015 and therefore CANNOT be relied on for stability of sorting. – peak Apr 03 '18 at 14:48
  • Thanks @peak I googled to install the latest package `sudo apt-get install jq`. Mine is debian. But I can't find out which version it will install. Is there any way to confirm the latest stable (please specify if you know) version before installing? – Naveen K Reddy Apr 04 '18 at 08:10
  • I believe apt-get will just get you jq 1.5. Since I don't know your platform, all I can do is suggest that either you use the workaround I've already mentioned, or consult the jq Installation pages. – peak Apr 04 '18 at 08:26
  • @peak While reading it's looks like pretty much I understood it. But I couldn't tweak it for my requirements. :( I have added a new question https://stackoverflow.com/questions/49647815/jq-group-by-and-increment. Will you be able to check please? – Naveen K Reddy Apr 04 '18 at 09:47