Compare two json files using jq or any other tools in bash

Question

I want to compare two json files to see if one can be extracted from the other one.

P1 (p1.json)

{
  "id": 12,
  "keys": ["key1","key2"],
  "body": {
    "height": "180cm",
    "wight": "70kg"
  },
  "name": "Alex"
}

P2 (p2.json)

{
  "id": 12,
  "keys": ["key2","key1"],
  "body": {
    "height": "180cm"
  }
}

As it can be seen P2 is not completely equal to P1 but it can be extracted from P1 (It provides less data about the same person but the data is correct).

Expected behavior:

p1 extends p2 --> true
p2 extends p1 --> false

Notes
- An array cannot be extracted from the same array with some additional elements

peak · Answer 1 · 2017-10-17T14:55:29.203

The following definition of extends/1 uses a purely object-based definition of extension (in particular, it does not sort arrays). The OP requirements regarding arrays are unclear to me, but a variant definition is offered in the following section.

# Usage: $in | extends($b) iff $in contains $b in an object-based sense
def extends($b):
  # Handle the case that both are objects:
  def objextends($x):
    . as $in | all($x|keys[]; . as $k | $in[$k] | extends($x[$k]));
  # Handle the case that both are arrays:
  def arrayextends($x):
    . as $in
    | length == ($x|length) and
        all( range(0;length); . as $i | $in[$i] | extends($x[$i]));

  if . == $b then true
  else . as $in
  | type as $intype
  | ($intype == ($b|type)) and
      (($intype == "object" and objextends($b)) or
       ($intype == "array" and arrayextends($b)))

end;

Examples:

{a:{a:1,b:2}, b:2} | extends({a:{a:1}}) # true

{a:{a:1,b:2}, b:2} | extends({a:{a:2}}) # false

{a:{a:1,b:2}, b:[{x:1,y:2}]} | extends({a:{a:2}, b:[{x:1}]}) # true

Alternative definition

The following definition sorts arrays and is sufficiently generous to handle the given example:

# Usage: $in | extends2($b) iff $in contains $b in a way which ignores the order of array elements
def extends2($b):
  # Both are objects
  def objextends($x):
    . as $in | all($x|keys[]; . as $k | $in[$k] | extends($x[$k]));

  def arrayextends($x): ($x|sort) - sort == [];

  if . == $b then true
  else . as $in
  | type as $intype
  | ($intype == ($b|type)) and
      (($intype == "object" and objextends($b)) or
       ($intype == "array"  and arrayextends($b)))
  end;

With $P1 and $P2 as shown:

  $P1 | extends2($P2) # yields true

Programming in `jq` seems bordering on the perverse. Gotta learn me some of that. :) — randomir, Oct 17 '17 at 14:21

score 0 · Answer 2 · answered Oct 17 '17 at 18:50

If you know there are no duplicates in any subarrays then you could use this approach which computes the difference between sets of [path,value] pairs returned from tostream replacing array indices with null:

def details:[
     tostream
   | select(length==2) as [$p,$v]
   | [$p|map(if type=="number" then null else . end),$v]
];

def extends(a;b): (b|details) - (a|details) == [];

If P1 and P2 are functions returning the sample data

def P1: {
    "id": 12,
    "keys": ["key1","key2"],
    "body": {
      "height": "180cm",
      "wight": "70kg"
    },
    "name": "Alex"
  }
;

def P2: {
    "id": 12,
    "keys": ["key2","key1"],
    "body": {
      "height": "180cm"
    }
  }
;

then

  extends(P1;P2)  # returns true
, extends(P2;P1)  # returns false

In the presence of duplicates the result is less clear. e.g.

  extends(["a","b","b"];["a","a","b"])  # returns true

Try it online!

Compare two json files using jq or any other tools in bash

2 Answers2

Alternative definition