1

I have a simple json

{
    "years": [ "2018", "2019" ]
}

that I want to read with Benthos. However as this is a "beautified" json, Benthos reads each line as message. How can one read this multiline json with benthos and compactify it for further downstream processors?

input:
  file:
    paths: [./init.json]
    codec: lines/multipart

pipeline:
  processors: ??

output:
  type: stdout

The pipeline should simply print the same in stdout as jq -c ./init.json would.

nerdizzle
  • 424
  • 4
  • 17

1 Answers1

1

try this:

input:
  file:
    paths: [./init.json]
    codec: all-bytes

  processors:
    - mapping: |
        root = this.format_json()

output:
  stdout:
    codec: all-bytes

Also, note that the current release is V4, where type: stdout isn't how you specify the output. Details here.

Mihai Todor
  • 8,014
  • 9
  • 49
  • 86
  • 1
    Ah thank you for the answer and the hint about concerning the output! I just add the line ` root = root.parse_json()` to the mapping such that I have the compactified json in a line. But I will accept your answer :) – nerdizzle Apr 20 '23 at 16:59
  • 1
    The `this` keyword does JSON parsing implicitly if the input data is valid JSON. Otherwise, you get an error. I added `format_json()` so you get a pretty-printed JSON output. – Mihai Todor Apr 20 '23 at 17:02
  • 1
    thank you for the hint! so its simply `root = this`. – nerdizzle Apr 20 '23 at 17:12
  • Yeah, that's the identity transform. In your case, it's helpful to have it if you want to validate the input JSON. Otherwise, it just passes it along to the output as raw bytes – Mihai Todor Apr 20 '23 at 17:37