Nested transformation using jolt transformer

Question

How can I do nested / recursive transformation using Jolt transformation. I m trying to the get the desired output before inputting it to the subsequent service.

My Input is

            {
                  "took": 7,
                  "timed_out": false,
              "_shards": {
                "total": 1,
                "successful": 1,
                "failed": 0
              },
              "hits": {
                "total": 5,
                "max_score": 1.000438,
                "hits": [
                  {
                    "_id": "AV-SJgvFPkCspwtrqHA1",
                    "_source": {
                      "tenant_id": "tenant1",
                      "session_id": "e780ff74-d33e-4024-9bb7-971f067484ea"
                    },
                    "inner_hits": {
                      "network_events": {
                        "hits": {
                          "total": 1,
                          "max_score": 6.0892797,
                          "hits": [
                            {
                              "_source": {
                                "event_id": 16,
                                "response_time": 0,
                                "url": "http://www.google.com/"
                              }
                            },
                            {
                              "_source": {
                                "event_id": 18,
                                "response_time": 1,
                                "url": "http://www.google1.com/"
                              }
                            }
                          ]
                        }
                      }
                    }
                  },
                  {
                    "_id": "BS-SJgvFPkCspwtrqHA1",
                    "_source": {
                      "tenant_id": "tenant2",
                      "session_id": "f4939272-d33e-4024-9bb7-971f067484ea"
                    },
                    "inner_hits": {
                      "network_events": {
                        "hits": {
                          "total": 1,
                          "max_score": 6.0892797,
                          "hits": [
                            {
                              "_source": {
                                "event_id": 18,
                                "response_time": 4,
                                "url": "http://www.google4.com/"
                              }
                            },
                            {
                              "_source": {
                                "event_id": 5,
                                "response_time": 5,
                                "url": "http://www.google5.com/"
                              }
                            }
                          ]
                        }
                      }
                    }
                  }
                ]
              }
            }

And the desired output is

{
   “sessions”: [
      {
            session_id : “S1”,
            tenant_id : “T1”,
            network_events : [
                {
                   “url”: “A”,
                   “response_time” : 22
                },
                {
                   “url”: “B,
                   “response_time” : 1
                }          
           ]


      },
      {
            session_id : “S2”,
            tenant_id : “T1”,
            network_events : [
                {
                   “url”: “C”,
                   “response_time” : 22
                }
           ]


      }
]
}

Is this possible using Jolt. I tried multiple combination using the sample examples but didn't get much.

I am new to Jolt so any help will be appreciated.

score 2 · Answer 1 · edited Dec 16 '17 at 10:15

[
  {
    "operation": "shift",
    "spec": {
      "hits": {
        "hits": {
          "*": {
            "_source": {
              "tenant_id": "sessions[#3].tenant_id",
              "session_id": "sessions[#3].session_id"
            },
            "inner_hits": {
              "network_events": {
                "hits": {
                  "hits": {
                    "*": {
                      "_source": {
                        "@": "sessions[#8].network_events.[]"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "remove",
    "spec": {
      "sessions": {
        "*": {
          "network_events": {
            "*": {
              "data_in": "",
              "data_out": "",
              "attributes": "",
              "event_id": "",
              "parent_url": "",
              "resource_type": ""
            }
          }
        }
      }
    }
  }
]

score 0 · Answer 2 · answered Sep 15 '22 at 15:03

You may consider another library.

https://github.com/octomix/josson

Deserialization

Josson josson = Josson.fromJsonString(
    "{" +
    "    \"took\": 7," +
    "    \"timed_out\": false," +
    "    \"_shards\": {" +
    "        \"total\": 1," +
    "        \"successful\": 1," +
    "        \"failed\": 0" +
    "    }," +
    "    \"hits\": {" +
    "        \"total\": 5," +
    "        \"max_score\": 1.000438," +
    "        \"hits\": [" +
    "            {" +
    "                \"_id\": \"AV-SJgvFPkCspwtrqHA1\"," +
    "                \"_source\": {" +
    "                    \"tenant_id\": \"tenant1\"," +
    "                    \"session_id\": \"e780ff74-d33e-4024-9bb7-971f067484ea\"" +
    "                }," +
    "                \"inner_hits\": {" +
    "                    \"network_events\": {" +
    "                        \"hits\": {" +
    "                            \"total\": 1," +
    "                            \"max_score\": 6.0892797," +
    "                            \"hits\": [" +
    "                                {" +
    "                                    \"_source\": {" +
    "                                        \"event_id\": 16," +
    "                                        \"response_time\": 0," +
    "                                        \"url\": \"http://www.google.com/\"" +
    "                                    }" +
    "                                }," +
    "                                {" +
    "                                    \"_source\": {" +
    "                                        \"event_id\": 18," +
    "                                        \"response_time\": 1," +
    "                                        \"url\": \"http://www.google1.com/\"" +
    "                                    }" +
    "                                }" +
    "                            ]" +
    "                        }" +
    "                    }" +
    "                }" +
    "            }," +
    "            {" +
    "                \"_id\": \"BS-SJgvFPkCspwtrqHA1\"," +
    "                \"_source\": {" +
    "                    \"tenant_id\": \"tenant2\"," +
    "                    \"session_id\": \"f4939272-d33e-4024-9bb7-971f067484ea\"" +
    "                }," +
    "                \"inner_hits\": {" +
    "                    \"network_events\": {" +
    "                        \"hits\": {" +
    "                            \"total\": 1," +
    "                            \"max_score\": 6.0892797," +
    "                            \"hits\": [" +
    "                                {" +
    "                                    \"_source\": {" +
    "                                        \"event_id\": 18," +
    "                                        \"response_time\": 4," +
    "                                        \"url\": \"http://www.google4.com/\"" +
    "                                    }" +
    "                                }," +
    "                                {" +
    "                                    \"_source\": {" +
    "                                        \"event_id\": 5," +
    "                                        \"response_time\": 5," +
    "                                        \"url\": \"http://www.google5.com/\"" +
    "                                    }" +
    "                                }" +
    "                            ]" +
    "                        }" +
    "                    }" +
    "                }" +
    "            }" +
    "        ]" +
    "    }" +
    "}");

Transformation

JsonNode node = josson.getNode(
    "hits.hits" +
    ".map(_source.session_id," +
    "     _source.tenant_id," +
    "     netwrok_events: inner_hits.network_events.hits.hits._source.map(url, response_time))" +
    ".toObject('sessions')");
System.out.println(node.toPrettyString());

Output

{
  "sessions" : [ {
    "session_id" : "e780ff74-d33e-4024-9bb7-971f067484ea",
    "tenant_id" : "tenant1",
    "netwrok_events" : [ {
      "url" : "http://www.google.com/",
      "response_time" : 0
    }, {
      "url" : "http://www.google1.com/",
      "response_time" : 1
    } ]
  }, {
    "session_id" : "f4939272-d33e-4024-9bb7-971f067484ea",
    "tenant_id" : "tenant2",
    "netwrok_events" : [ {
      "url" : "http://www.google4.com/",
      "response_time" : 4
    }, {
      "url" : "http://www.google5.com/",
      "response_time" : 5
    } ]
  } ]
}

Nested transformation using jolt transformer

2 Answers2