is there any way to write a null json transformation (passes through orig document) using Jolt?

Question

You know how XSLT and other XML processing languages support the "null transformation" which passes a document through unmodified ?

I would like to do the same thing for Jolt (a very nice JSON transformation library used in Apache Camel and other places).

I could use JOLT's "insert default" feature and stick some harmless JSON tag and value at the top level of the document.. which is almost what want. But I couldnt' figure out how to pass through the document through JOLT but leave it untouched.

Why do i want to do this you ask ? We are developing a streaming data pipeline and I have to validate incoming strings as valid JSON... Jolt does that for me for free, but in some cases I don't want to monkey with the document. So, I want to use JOLT as a step in the pipeline, but (in some cases) have it do nothing to the input JSSON doc.

score 2 · Answer 1 · edited Aug 14 '19 at 06:57

Another option is to create a custom transformer.

package com.example;
public class NullTransform implements Transform{

    @Override
    public Object transform(Object input) {
        return input;
    }

}

then reference it from the chainr jolt as below

[
  {
  "operation": "com.example.NullTransform"
  }
]

You'll still incur the deserializaion/serialization overhead but no other code is run.

score 0 · Accepted Answer · answered Jan 22 '17 at 19:30

OOTB Jolt contains 5 "operations" that can be applied to the input hydrated Json. 4 of those (default, remove, sort, cardinality) are mutation operations / modify the supplied hydrated Json. I you gave those 4 an empty "spec", they would do nothing, and your data would "pass thru".

The "shift" operation does not mutate the input it is given. Instead it "copies" data from the "input" to a new "output" map/list. If you don't give "shift" a spec, then it copies nothing across.

Thus, from you question, it sounds like you are talking about "shift". With shift you have to explicitly pass thru all the things you "want to keep".

Depending on your data this may be terrible or easy, as you can have shift copy very large chunks of data across.

Example, for the "inception" example on the jolt demo site. http://jolt-demo.appspot.com/#inception

This Spec basically passes thru the input, coping the whole nested map that is "rating" thru to the output.

[
  {
    "operation": "shift",
    "spec": {
      "rating": "rating"
    }
  }
]

It can be generalized with wildcards to : Spec

[
  {
    "operation": "shift",
    "spec": {
      "*": "&"
    }
  }
]

Thanks again for the excellent support, Milo. Can I assume the wildcarded shift (* => &) is a pretty quick operation ? Specifically, do you think that would be as fast as injecting a standard key / value (e.g "version:" "1.2.3") at the top level of the json document using the "default" operation. we will go with the question transform. /cb — Chris Bedford, Jan 22 '17 at 19:55
A single "default" at the top level will be faster. The wildcarded shift (* => &) will have to visit / copy across each "key" and "value" from the input map. — Milo S, Feb 02 '17 at 02:48
got it. i know i'm not supposed to say thanks... But thanks ! — Chris Bedford, Feb 02 '17 at 21:06

is there any way to write a null json transformation (passes through orig document) using Jolt?

2 Answers2