jq add character before and after every row JSON

Question

I want print "[]" in every row with jq to make a big file json proof

I have:

{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}
{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}
{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}
{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}

What i want:

[[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]]

Now i can push the json line by line to a program.

The output shown is _not_ a valid JSON. Update your question to reflect the actual. Did you mean to show an array of arrays? — Inian, Sep 22 '22 at 14:23
The input is a nice stream of JSON objects. The output you indicate you want is not valid JSON, so it's hard to see why you describe it as "json proof". If you want to convert a stream of JSON entities to a single JSON array, which would make sense, you can just run `jq -s .`. There is hopefully no good reason to do anything more complicated. — peak, Sep 22 '22 at 16:20

score 0 · Answer 1 · answered Sep 22 '22 at 14:26

If you input are just plain objects:
Use --slurp to create a single array, then use map([ . ]) to wrap each object in his own array

jq --slurp --compact-output 'map([ . ])'
[[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}],[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}], ...

JqPlay demo

score 0 · Answer 2 · answered Sep 22 '22 at 14:52

Assuming your file contains a stream of stand-alone JSON objects, the filter is simply [.]. The filter will be applied to each input object.

jq -c '[.]' file.json

Output:

[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]

To wrap everything in one big array:

jq -s . file.json
# or:
jq -s <file.json

Output:

[
  {
    "test": "index",
    "test2": "fdsdfs",
    "test3": "dfs0D0WOQAA3"
  },
  {
    "test": "index",
    "test2": "fdsdfs",
    "test3": "dfs0D0WOQAA3"
  },
  {
    "test": "index",
    "test2": "fdsdfs",
    "test3": "dfs0D0WOQAA3"
  },
  {
    "test": "index",
    "test2": "fdsdfs",
    "test3": "dfs0D0WOQAA3"
  }
]

And to wrap each single item in an array and all arrays in a top-level array:

jq 'map([.])' file.json

Output:

[
  [
    {
      "test": "index",
      "test2": "fdsdfs",
      "test3": "dfs0D0WOQAA3"
    }
  ],
  [
    {
      "test": "index",
      "test2": "fdsdfs",
      "test3": "dfs0D0WOQAA3"
    }
  ],
  [
    {
      "test": "index",
      "test2": "fdsdfs",
      "test3": "dfs0D0WOQAA3"
    }
  ],
  [
    {
      "test": "index",
      "test2": "fdsdfs",
      "test3": "dfs0D0WOQAA3"
    }
  ]
]

score 0 · Answer 3 · answered Sep 22 '22 at 14:59

I assume you just forgot to put the commas in your otherwise valid JSON output, in which case you can find reasonable answers elsewhere on this page.

If not, and your desired output is accurate as shown, one way to solve it with jq is to read in the lines as raw text, prepend and append to certain lines as you see fit, and output again as raw text:

jq -Rnr '[inputs] | (first, .[]) |= "[" + . | (last, .[]) += "]" | .[]'

[[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]
[{"test":"index","test2":"fdsdfs","test3":"dfs0D0WOQAA3"}]]

Demo

But, to be fair, you actually don't need a JSON processor if it is just plain text being manipulated. Using sed or awk would be more reasonable choices here.

jq add character before and after every row JSON

3 Answers3