How to map using Jolttransform specification in NIFI?

Question

I am preparing the jolt specification but I am unable to complete it, pls help out.

Note: There is a credits array, the values of that array should be mapped with schema fields which are of type credit record.

Input:

{
  "schema": {
    "fields": [
      {
        "name": "user",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "billed",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "currency",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "currency_conversion_rate",
        "type": "FLOAT",
        "mode": "NULLABLE"
      },
      {
        "name": "usage_unit",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "adjustment_info_mode",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "adjustment_info_type",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "InvoiceMonth",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "credits",
        "type": "RECORD",
        "mode": "REPEATED",
        "fields": [
          {
            "name": "name",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "amount",
            "type": "FLOAT",
            "mode": "NULLABLE"
          },
          {
            "name": "full_name",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "id",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "type",
            "type": "STRING",
            "mode": "NULLABLE"
          }
        ]
      },
      {
        "name": "resource_name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "resource_global_name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "label",
        "type": "STRING",
        "mode": "NULLABLE"
      }
    ]
  },
  "InvoiceMonth": "202006",
  "credits": [
    {
      "v": {
        "f": [
          {
            "v": "ae391_credit:0"
          },
          {
            "v": "-0.010237"
          },
          {
            "v": "Committed dollar "
          },
          {
            "v": "ae31_credit:0"
          },
          {
            "v": "COMMITTED_BASE"
          }
        ]
      }
    },
    {
      "v": {
        "f": [
          {
            "v": "ae391_credit:0"
          },
          {
            "v": "-0.010237"
          },
          {
            "v": "Committed dollar "
          },
          {
            "v": "ae31_credit:0"
          },
          {
            "v": "COMMITTED_BASE"
          }
        ]
      }
    }
  ],
  "resource_name": "projects-0",
  "resource_global_name": "//compute.googleapis.com/",
  "label": ""
}

I have tried this jolt:

[
  {
    "operation": "shift",
    "spec": {
      "credits": {
        "*": {
          "*": {
            "*": {
              "*": {
                "@v": "[&3].@(5,schema.fields.fields[&].name)"
              }
            }
          }
        }
      }
    }
  }
]

Expected Output:

[
  {
    "name": "ae391_credit:0",
    "amount": "-0.010237",
    "full_name": "Committed dollar",
    "id": "ae31_credit:0",
    "type": "COMMITTED_BASE"
  },
  {
    "name": "ae391_credit:0",
    "amount": "-0.010237",
    "full_name": "Committed dollar",
    "id": "ae31_credit:0",
    "type": "COMMITTED_BASE"
  }
]

score 1 · Answer 1 · answered Jun 12 '23 at 09:45

You can use a shift operation and bring inner fields up and use another shift for getting the value of the name as your key in the credits array.

[
  {
    "operation": "shift",
    "spec": {
      "credits": "&",
      "schema": {
        "fields": {
          "*": {
            "fields": "&"
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "credits": {
        "*": {
          "*": {
            "*": {
              "*": {
                "*": "[&4].@(6,fields[&1].name)"
              }
            }
          }
        }
      }
    }
  }
]

How to map using Jolttransform specification in NIFI?

1 Answers1