2

I am trying with Nifi JOLT processor to shift the flat JSON object to a list of JSON object as in below output. The params "p_7_1_0", "px_2_7_1_0", "pv_7_1_1" may be different in names or numbers (e.g. I could have { "timestamp": 1559347670, "pw_2_1_0": 1, "p_2_2_1_0": 1 } )

Could someone help me with the jolt specs?

Input Json:

{
  "timestamp": 1559347670,
  "p_7_1_0": 6,
  "px_2_7_1_0": 1,
  "pv_7_1_1": 1
}

Expected output JSON:

{
  "values": [
    {
      "key": "p_7_1_0",
      "value": 6,
      "timestamp": 1559347670
    },
    {
      "key": "px_2_7_1_0",
      "value": 1,
      "timestamp": 1559347670
    },
    {
      "key": "pv_7_1_1",
      "value": 1,
      "timestamp": 1559347670
    }
  ]
}

Thanks in advance

Barbaros Özhan
  • 59,113
  • 10
  • 31
  • 55
Alex
  • 27
  • 5

2 Answers2

2

After reading up on this question JOLT transformation to copy single value along an array

and the answer https://stackoverflow.com/a/50438480/2733184

I can see that what you want is eerily similar. However, I would have never hit the nail on the head with the question that needed to be asked.

I encourage you to go to the aforementioned Q and the A, read all of it (including the comments inside the spec) and give them some upvotes.

[
  {
    "operation": "shift",
    "spec": {
      "timestamp": "timestamp",
      // put everything but the timestamp in a member (the pivot)
      "*": "all.&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "all": {
        "*": {
          // grab the member key and put it in its final place
          "$": "values[#2].key",
          // grab the member value and put it in its final place
          "@": "values[#2].value",
          // Walk back two steps (* -> all -> root) and grab the timestamp
          "@(2,timestamp)": "values[#2].timestamp"
          // I wish I understood the logic behind "#2" but I don't
          // and I'll have to read on it further
        }
      }
    }
  }
]

I hope someone can explain what # is for. My immediate guess is that it is like & (the member name) but it looks like it is the member position (?).

hanzo2001
  • 1,338
  • 1
  • 10
  • 24
0

You just can loop through the attributes having _ character to distinguish from timestamp within a single shift transformation spec such as

[
  {
    "operation": "shift",
    "spec": {
      "*_*": {
        "@1,timestamp": "values[#2].timestamp",
        "$": "values[#2].key",
        "@": "values[#2].value"
      }
    }
  }
]

where

  • "@1,timestamp" represents grabbing the value of timestamp attribute after going one level up the tree
  • by [#2] on the right-hand-side would logically mean traversing : and { as summed up 2 levels to reach the level of the topmost attributes while [# ] will generate arraywise results
Barbaros Özhan
  • 59,113
  • 10
  • 31
  • 55