JOLT transformation to copy single value along an array

Question

I want to transform a JSON into key-value pairs and simultaneously copy a specific single value, i.e. the "timestamp", to all these pairs.

Input JSON:

{
  "Timestamp": "2018-05-13T14:57:09",
  "first_key": "1023",
  "another_key": "1987",
  "yet_another_key": "677"
}

Expected output:

[
  {
    "Timestamp": "2018-05-13T14:57:09.087",
    "key": "first_key",
    "value": "1023"
  },
  {
    "Timestamp": "2018-05-13T14:57:09.087",
    "key": "another_key",
    "value": "1987"
  },
  {
    "Timestamp": "2018-05-13T14:57:09.087",
    "key": "yet_another_key",
    "value": "677"
  }
]

What I came up with so far is the following JOLT specification. It already generates the key-value pairs for all entries that are not "Timestamp", but how can I copy the value of "Timestamp" into each of these records?

[
  {
    "operation": "shift",
    "spec": {
      "Timestamp": "[].Timestamp",
      "*": {
        "$": "[#2].key",
        "@": "[#2].value"
      }
    }
  }
]

Output of the above JOLT specification:

[
  {
    "Timestamp": "2018-05-13T14:57:09.087"
  },
  {
    "key": "first_key",
    "value": "1023"
  },
  {
    "key": "another_key",
    "value": "1987"
  },
  {
    "key": "yet_another_key",
    "value": "677"
  }
]

score 3 · Accepted Answer · answered May 20 '18 at 19:09

Spec

[
  {
    // first separate the Timestamp from the fields that 
    //  are going to be pivoted.
    // This is needed because the "[#2]" logic 
    //  isn't doing what you think it is / 
    //  you would end up with a null in your output array.
    // Basically the "[#2]" logic as written only works
    //  when you are cleanly pivoting data.  Which you are
    //  not because "Timestamp" and "first_key" are siblings.
    "operation": "shift",
    "spec": {
      "Timestamp": "Timestamp",
      "*": "keysToPivot.&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "keysToPivot": {
        "*": {
          // Now that we can cleanly loop thru _only_ the 
          //  keysToPivot, the [#2] logic will work "right".
          // Additionally, lookup the Timestamp value
          //  and add it to each "pivoted" output.
          "$": "[#2].key",
          "@": "[#2].value",
          "@(2,Timestamp)": "[#2].Timestamp"
        }
      }
    }
  }
]

score 0 · Answer 2 · answered Mar 13 '23 at 12:16

You just can loop through the attributes ending with _key within a shift transformation spec such as

[
  {
    "operation": "shift",
    "spec": {
      "*_key": {
        "@1,Timestamp": "[#2].Timestamp",
        "$": "[#2].key",
        "@": "[#2].value"
      }
    }
  }
]

where "@1,Timestamp" represents grabbing the value of Timestamp attribute after traversing one level the tree

the demo on the site http://jolt-demo.appspot.com/ is

JOLT transformation to copy single value along an array

2 Answers2

Linked