1

I am trying to build a Excecute Script Processor for Nifi. It handles a JSON file, splits it and sends it to the next processor, which is an MongoDB writer. The logic works so far. The main problem is, that I cannot get the processor to create and send a new FlowFile for each new JSON created out of the input JSON. I got it working a bit but unfortunately, all the FlowFiles come out empty. Is there something wrong with the flow (from creation of a new Flow to sending it)?

Here is a code snippet:

var flowFile = session.get();
if (flowFile != null) {
    var StreamCallback = Java.type("org.apache.nifi.processor.io.StreamCallback");
    var IOUtils = Java.type("org.apache.commons.io.IOUtils");
    var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");
    try {

        flowFile = session.write(flowFile,
            new StreamCallback(function (inputStream, outputStream) {
                var content = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
                var json = JSON.parse(content);
                var events = json["events"];
                var mongoEvent = "";
                var flowFileList = [];

                for(var x = 0; x < json["events"].length; x++){
                    try{
                        var newFlowFile = session.create();
                        mongoEvent = constructJSONEvent(x, json); // Here we will receive our new JSON 
                        outputStream.write(mongoEvent.getBytes(StandardCharsets.UTF_8));

                        session.transfer(newFlowFile, REL_SUCCESS);

                    }catch(e){
                        session.transfer(newFlowFile, REL_FAILURE);
                    }
                }
            }));
        session.transfer(flowFile, REL_SUCCESS);
    } catch(e) {
        session.transfer(flowFile, REL_FAILURE);
    }
}
Derek Haynes
  • 125
  • 1
  • 2
  • 13
  • U r creating new flowfile `var newFlowFile = session.create();` but writing to the original flow file `outputStream.write(mongoEvent.getBytes(StandardCharsets.UTF_8))` – daggett Sep 06 '19 at 12:47
  • So I would need a new session.write() I suppose to send it? Isnt it overwriting the input stream then? – Derek Haynes Sep 07 '19 at 12:30
  • `session.create()` creates an empty file. just ignore input stream for child writers. or use `OutputStreamCallback` that accepts only output stream param. ps: in groovy code will be shorter. – daggett Sep 07 '19 at 13:24

0 Answers0