Exploring JOLT functions for JSON to JSON transformations: An overview

Question

Can someone give a brief explanation of different functions in the JOLT transformation library for JSON to JSON transformations?

A brief explanation of JOLT function.

---

• Like :- Function name - what it does -- its arguments

Answer 1

Brief overview of JOLT functions

String

• toLower - converts the string into lowercase -- can accept one argument that is string and if there are multiple args of string then it return an array.
• toUpper - converts the string into uppercase -- can accept one argument that is string and if there are multiple args of string then it return an array.
• concat - concats strings -- accepts multiple args of string to be concatenated.
• join - Join the string with the string given in first arg -- accepts multiple args of string or you can pass an array.
• split - split the string into an array based on the given delimiter -- accepts two args : first - delimiter, second - string
• substring - to create a substring -- accepts three args: first - string, second-start index, third - end index
• trim - trims the extra space before and after the string -- can accept one argument that is string and if there are multiple args of string then it return an array.
• leftPad e rightPad - adds the characters to the string on start or end to make it to the desired length -- accepts three args: first - input string, second - string desired length, third - character that is to be added

Type

• toInteger - converts to int
• toDouble - converts to double
• toLong - converts to long
• toBoolean - converts to int
• toString - converts to string
• squashNulls and recursivelySquashNulls - Regarding the recursivelySquashNulls and squashNulls functions, both are applicable only to objects and lists and serve to remove fields with null values, however while recursivelySquashNulls will look at all levels below the object or list, squashNulls will look just 1 level below.
• size - return the size of string

List

• firstElement - returns the first element from the array -- accepts only one arg i.e. array
• lastElement - returns the last element from the array -- accepts only one arg i.e. array
• elementAt - return the element at index -- accepts two args: first - array, second - index
• toList - return an array containing the given string -- accepts only one arg that is string
• sort - sorts the array in lexicographical order -- accepts only one arg i.e. array

Number

• min - return min from the array or values -- accepts multple args of vals or an array.
• max - returns max from the array or values -- accepts multple args of vals or an array.
• abs - give the absolute value -- accepts only one arg i.e. int val
• avg - calculates the avg from the array or vals-- accepts array or multiple vals
• intSum - sums the array or values -- accepts array or multiple vals
• doubleSum - sums the array or values returns the value in double -- accepts array or multiple vals
• longSum - sums the array or values returns the value in long -- accepts array or multiple vals
• intSubtract - subtracts two value -- accepts two arg - first and second val
• doubleSubtract -- accepts two arg - first and second val
• longSubtract -- accepts two arg - first and second val
• divide - divides two value -- accepts two arg - first and second val
• divideAndRound - divides the value and round it upto the given decimal places - accepts three arg: first - the decimal places, second - dividend, third - divisor

Answer 2

Here is example of each function

JSON input

{
  "STRING": {
    "product": "Product A",
    "company": "company a",
    "value": "100",
    "measureWithSpaces": "  10 meters "
  },
  "NUMBER": {
    "array": [ 3, 5, 2, 7, 1 ],
    "negativeValue": -100,
    "positiveValue": 50
  },
  "TYPE": {
    "value": 10.5,
    "stringBoolean": "true",
    "objectWithNull": {
      "fielWithValue": "ABC",
      "nullField": null
    }
  },
  "LIST": {
    "array": [ "c", "t", "m", "a" ],
    "stringField": "123"
  }
}

Jolt Specification

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "STRING": {
        "product": "=toLower(@(1,product))",
        "company": "=toUpper(@(1,company))",
        "product_company": "=concat(@(1,product),'_',@(1,company))",
        "joinProductCompany": "=join(' - ',@(1,product),@(1,company))",
        "splitProductCompany": "=split('[-]',@(1,joinProductCompany))",
        "substringProduct": "=substring(@(1,product),0,4)",
        "value": "=leftPad(@(1,value),6,'A')",
        "measure": "=trim(@(1,measureWithSpaces))"
      },
      "NUMBER": {
        "minArray": "=min(@(1,array))",
        "maxArray": "=max(@(1,array))",
        "absoluteValue": "=abs(@(1,negativeValue))",
        "averageArray": "=avg(@(1,array))",
        "sumArray": "=intSum(@(1,array))",
        "subtrArray": "=intSubtract(@(1,positiveValue), 20)",
        "division": "=divide(@(1,positiveValue),2)",
        "divisionRound": "=divideAndRound(3,@(1,positiveValue),3)"
      },
      "TYPE": {
        "integerValue": "=toInteger(@(1,value))",
        "booleano": "=toBoolean(@(1,stringBoolean))",
        "stringValue": "=toString(@(1,value))",
        "stringBoolean": "=size",
        "objectWithNull": "=recursivelySquashNulls"
      },
      "LIST": {
        "arrayFirstItem": "=firstElement(@(1,array))",
        "arrayLastItem": "=lastElement(@(1,array))",
        "arrayElement": "=elementAt(@(1,array),2)",
        "fieldToList": "=toList(@(1,stringField))",
        "orderedArray": "=sort(@(1,array))"
      }
    }
  }
]

JSON output

{
  "STRING": {
    "product": "product a",
    "company": "COMPANY A",
    "value": "AAA100",
    "measureWithSpaces": "  10 meters ",
    "product_company": "product a_COMPANY A",
    "joinProductCompany": "product a - COMPANY A",
    "splitProductCompany": [
      "product a ",
      " COMPANY A"
    ],
    "substringProduct": "prod",
    "measure": "10 meters"
  },
  "NUMBER": {
    "array": [ 3, 5, 2, 7, 1 ],
    "negativeValue": -100,
    "positiveValue": 50,
    "minArray": 1,
    "maxArray": 7,
    "absoluteValue": 100,
    "averageArray": 3.6,
    "sumArray": 18,
    "subtrArray": 30,
    "division": 25,
    "divisionRound": 16.667
  },
  "TYPE": {
    "value": 10.5,
    "stringBoolean": 4,
    "objectWithNull": {
      "fielWithValue": "ABC"
    },
    "integerValue": 10,
    "booleano": true,
    "stringValue": "10.5"
  },
  "LIST": {
    "array": [
      "c",
      "t",
      "m",
      "a"
    ],
    "stringField": "123",
    "arrayFirstItem": "c",
    "arrayLastItem": "a",
    "fieldToList": [
      "123"
    ],
    "orderedArray": [ "a", "c", "m", "t" ]
  }
}