How can I replace a value in json data in Java?

Question

I am trying to replace API Data with "All" in Emp_Id field in data in json. And then make rows with every data of API. I tried in ecmascript but I need in Java as this map function is giving error in nifi.

API DATA: I have stored this data in "Response" attribute in ExtractText in Apache Nifi.

response 
{
    "status": "success",
    "data": [[123, 0], [124, 0], [446, 0], [620, 0], [470 ,1]]
};

jsonData
{
    "Emp_Id": "All",
    "Emp_loc": "523",
    "Emp_dept": "Management",
    "Emp_sub_dept": "Finance",
    "Emp_sub_dept2": "Accountant"
};

Expected Result

[
{
"Emp_Id":"123",
"Emp_loc":"523",
"Emp_dept":"Management",
"Emp_sub_dept":"Finance",
"Emp_sub_dept2":"Accountant"
},
{
"Emp_Id":"124",
"Emp_loc":"523",
"Emp_dept":"Management",
"Emp_sub_dept":"Finance",
"Emp_sub_dept2":"Accountant"
},
{
"Emp_Id":"446",
"Emp_loc":"523",
"Emp_dept":"Management",
"Emp_sub_dept":"Finance",
"Emp_sub_dept2":"Accountant"
},
{
"Emp_Id":"620",
"Emp_loc":"523",
"Emp_dept":"Management",
"Emp_sub_dept":"Finance",
"Emp_sub_dept2":"Accountant"
},
{
"Emp_Id":"470",
"Emp_loc":"523",
"Emp_dept":"Management",
"Emp_sub_dept":"Finance",
"Emp_sub_dept2":"Accountant"
}
]

This is tried in ecmaScript but I want it to be in Java because these functions are not working in executeScript of NIFI and giving error."java.lang.assertionError: geberating bytecode". Or if any other approach for converting this data.

this is the script I tried

var InputStreamCallback = Java.type("org.apache.nifi.processor.io.InputStreamCallback")
var IOUtils = Java.type("org.apache.commons.io.IOUtils");
var OutputStreamCallback =  Java.type("org.apache.nifi.processor.io.OutputStreamCallback");
var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");
var Set = Java.type("java.util.HashSet");
var Map = Java.type("java.util.HashMap");
var String = Java.type("java.lang.String");

var flowFile = session.get();
    
if (flowFile != null) {  
    var text = ''
    session.read(flowFile,
    new InputStreamCallback(function (inputStream) {
    text = IOUtils.toString(inputStream, StandardCharsets.UTF_8);             
}));

var s3Data ={}
var apiResponse={}
var map = new Map();
    
var s3Data = JSON.parse(flowFile.getAttribute('jsonData'))
var apiResponse = JSON.parse(flowFile.getAttribute('response'))

var result = apiResponse.data.map(([id]) =>
Object.fromEntries(Object.entries(s3Data).map(([k, v]) =>
[k, v === "All" ? id : v]
))
);  
flowFile = session.write(flowFile,
    new OutputStreamCallback(function(outputStream) {
    outputStream.write(JSON.stringify(result).getBytes(StandardCharsets.UTF_8))
})
);
}

I tried with groovy also but I have very less knowledge of Groovy so I couldn't write the proper ExecuteScript.

No they dont, but data is like this only, So I have to ignore the 0's and 1's. And I forgot to add for 617, it goes same as of other indexe values. — dash, May 29 '23 at 16:55

score 1 · Answer 1 · answered Jun 04 '23 at 18:22

You can try converting it to JSONObject and then replace it with the desired values.

Parse response String to simple.JSONObject
get the data value out of it
convert it to list of only first value of data.
loop through the list and replace the jsonData's emp_id with new value.

String response = "{\r\n"
                + "    \"status\": \"success\",\r\n"
                + "    \"data\": [[123, 0], [124, 0], [446, 0], [620, 0], [470 ,1]]\r\n"
                + "}";
        String jsonData = "{\r\n"
                + "    \"Emp_Id\": \"All\",\r\n"
                + "    \"Emp_loc\": \"523\",\r\n"
                + "    \"Emp_dept\": \"Management\",\r\n"
                + "    \"Emp_sub_dept\": \"Finance\",\r\n"
                + "    \"Emp_sub_dept2\": \"Accountant\"\r\n"
                + "}";
        JSONParser parser = new JSONParser();
        JSONObject respObj = (JSONObject)parser.parse(response);
        List<JSONArray> numberList = (List)respObj.get("data");
        List<Long> replacableNumberList = numberList.stream().map(jsonArray -> (Long)jsonArray.get(0))
                .collect(Collectors.toList());
        
        
        List<JSONObject> outputList = new ArrayList<>();
        for (Long integer : replacableNumberList) {
            //if parsing before loop then dont forget to create new object before adding in the list
            JSONObject jsonObj = (JSONObject)parser.parse(jsonData);
            jsonObj.put("Emp_Id", integer);
            outputList.add(jsonObj);
        }
        outputList.forEach(System.out::println);

score 0 · Answer 2 · answered Jun 04 '23 at 17:12

You may try library Josson & Jossons to transform and join the two datasets.

https://github.com/octomix/josson

Deserialize, Transform and Join

Left join <=< two transformed datasets with matching key.

Josson response = Josson.fromJsonString(
    "{" +
    "    \"status\": \"success\"," +
    "    \"data\": [[123, 0], [124, 0], [446, 0], [617, 1], [620, 0], [470 ,1]]" +
    "}");
Josson jsonData = Josson.fromJsonString(
    "{" +
    "    \"Emp_Id\": \"All\"," +
    "    \"Emp_loc\": \"523\"," +
    "    \"Emp_dept\": \"Management\"," +
    "    \"Emp_sub_dept\": \"Finance\"," +
    "    \"Emp_sub_dept2\": \"Accountant\"" +
    "}");
JsonNode node = new Jossons()
    .putDataset("response", response)
    .putDataset("jsonData", jsonData)
    .evaluateQuery("response->data@.[0].map(Emp_Id:?,key:'All'){key} <=< jsonData->field(key:Emp_Id,Emp_Id:){key}");
System.out.println(node.toPrettyString());

Output

[ {
  "Emp_Id" : 123,
  "key" : "All",
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant"
}, {
  "Emp_Id" : 124,
  "key" : "All",
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant"
}, {
  "Emp_Id" : 446,
  "key" : "All",
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant"
}, {
  "Emp_Id" : 617,
  "key" : "All",
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant"
}, {
  "Emp_Id" : 620,
  "key" : "All",
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant"
}, {
  "Emp_Id" : 470,
  "key" : "All",
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant"
} ]

Demonstrate the transformation of response

Divert each data element to separate branches. For each branch, get the first array element and then build an object with element Emp_Id and key.

JsonNode node = response.getNode("data@.[0].map(Emp_Id:?,key:'All')");
System.out.println(node.toPrettyString());

Output

[ {
  "Emp_Id" : 123,
  "key" : "All"
}, {
  "Emp_Id" : 124,
  "key" : "All"
}, {
  "Emp_Id" : 446,
  "key" : "All"
}, {
  "Emp_Id" : 617,
  "key" : "All"
}, {
  "Emp_Id" : 620,
  "key" : "All"
}, {
  "Emp_Id" : 470,
  "key" : "All"
} ]

Demonstrate the transformation of jsonData

Add an element key with the value of Emp_Id and then remove Emp_Id.

JsonNode node = jsonData.getNode("field(key:Emp_Id,Emp_Id:)");
System.out.println(node.toPrettyString());

Output

{
  "Emp_loc" : "523",
  "Emp_dept" : "Management",
  "Emp_sub_dept" : "Finance",
  "Emp_sub_dept2" : "Accountant",
  "key" : "All"
}

How can I replace a value in json data in Java?

2 Answers2