0

Input

{
"item" : "some value",
"get"  : :some other value",
"org"{
"animal-type": ["mammal","reptile","bird"],
"animal-name": ["rat","pigeon"],
"some-random": ["one array"]
....
....any number of items with any name can show up here.. 
.... 
....
},
"some_weight" : "yuad asf-do-not-convet"
}

Output

{
"item" : "some value",
"get"  : :some other value",
....
Array of "org"
....
.... any format with "-" converted to "_" AND any representation in such a way that the number of elements in it is a fixed number
.... This one has ultimately to be converted into avro schema and we don't want to update schemas when occasional items appear within "org"
....
"some_weight" : "yuad asf-do-not-convet"
}

Jolt I have been trying so far,

[
 {
"operation": "shift",
"spec": {
      "org": {
        "*-*-*": "&(0,1)_&(0,2)_&(0,3)",
        "*-*": "&(0,1)_&(0,2)",
        "*": "&"
      },
      "*": "&"
    }
  }
]

This replaces the - to _ and removes the org, which is totally fine with me. However, I am struggling how to make the names, animal-type,animal-name,some-random (three here but can be any number) into a key value type representation so that I don't have to keep adjusting my avro schema with some fields that I am not anticipating appear.

OpensourceEnthu
  • 31
  • 1
  • 4
  • 1
    It's unclear what your are asking. You have some JSON you want to transform, but you have some specific requirements. What have you tried? No one is really going to write any of this because it depends on so many other things we don't know about. Well, that, and no one really wants to write code for free here. –  Sep 20 '18 at 14:58
  • I have no idea what type of jolt transform will work here. I have used shift operation, but I am not even aware if there is any way to deal with this complicated type of jolt transform to convert object type to array. Thank you for looking into it though – OpensourceEnthu Sep 20 '18 at 16:16
  • This is the sort of thing that belongs in the text of the question, not comments. –  Sep 20 '18 at 18:43
  • Please add the following to your question 1) input json 2) output json that you are expecting and 3) the JOLT transform that you are using. And the problem you have with it. – user4321 Sep 21 '18 at 14:34

0 Answers0