How to compress output of arrays of primitive type?

Question

My json file contains arrays of ints and strings and objects. Is there a way of compressing the output of arrays that contain only ints or strings?

either do not display elements of these arrays, or
show type of elements and count

This is what it looks like now:

$ jq "." foo.json
{
    "version": [
            "2.53.0",
            "2.53.0",
            "2.53.0",
            "2.53.0",
            "2.53.0",
            "2.53.3",
            "2.53.3",
            "2.53.0",
            "2.53.0",
            "2.53.3",
            "2.53.0",
            "2.53.0",
            "2.53.3",
            "2.53.0",
            "2.53.0",
            "2.53.0",
            "2.53.0",
            "2.53.0",
            "2.53.3",
            "2.53.0"
          ],
    "walltime_seconds": [
            0.165,
            0.199,
            0.415,
            0.193,
            12.114,
            0.227,
            12.341,
            12.145,
            0.135,
            0.326,
            0.293,
            0.19,
            0.271,
            0.103,
            0.196,
            0.18,
            0.177,
            0.166,
            0.506,
            0.568
          ]
}

This is what I would like:

{
    "version": "[..]",
    "walltime_seconds": "[..]",

}

or this

{
    "version": "Array(str, 20)",
    "walltime_seconds": "Array(float, 20)",

}

Of course the compression should happen anywhere in the json tree and it should only be done for int, str, float and not for objects.

The outputs you have shown are not valid JSON – Inian Apr 22 '22 at 17:35 — Inian, Apr 22 '22 at 17:35
@Inian good point, I changed it. – Leevi L Apr 22 '22 at 18:31 — Leevi L, Apr 22 '22 at 18:31

peak · Accepted Answer · 2022-04-22T20:18:04.550

3

This will do it:

def compress:
  if type == "array"
  then (if    all(.[]; type == "string") then "[string]"
        elif  all(.[]; type == "number") then "[number]"
        elif  all(.[]; type == "boolean") then "[boolean]"
        else .
        end)
  else .
  end;

walk(compress)

With your sample input, the result would be:

{
  "version": "[string]",
  "walltime_seconds": "[number]"
}

To include the length and handle arrays of null:

def compress:
  def check($t):
    if all(.[]; type == $t) then "[\($t)[\(length)]]" else empty end;
  if type == "array"
  then check("string") // check("number") // check("boolean") // check("null") // .
  else .
  end;

edited Apr 22 '22 at 20:18

answered Apr 22 '22 at 17:44

peak

105,803
17
152
177

That's great, it works! Is there a way to store functions like `compress` somewhere? – Leevi L Apr 22 '22 at 18:39
One possibility would be to include it in a file: ~/.jq – peak Apr 22 '22 at 19:48
that's so cool, works! Thank's a lot! – Leevi L Apr 22 '22 at 19:51

How to compress output of arrays of primitive type?

1 Answers1