Questions tagged [flowfile]
29 questions
0
votes
0 answers
Find Provenance Data For Flowfile Within a Processor
I am attempting to develop a NiFi processor that would extend the functionality of the built-in processor "Monitor Activity".
The problem I am attempting to solve is that in my application, I would have multiple flows entering the processor, with…
tmus
- 1
0
votes
0 answers
Failed to write flowfile to stdin due to the pipe has been ended
I have been trying to get an excel file and process it using a python script in executestreamcommand. but it is showing this error.
Any idea on how to read excel file to stdin?
import pandas as pd
import openpyxl
df =…
Aakarsh Raj
- 65
- 9
0
votes
3 answers
Read flow file attribute/content to processor property
I want to set a property of a processor based on the contents of the last flowfile that came through.
Example: I instantiate the flowfile with the processor GenerateFlowFile and with the custom text ${now()} as the current timestamp during the…
Cribber
- 2,513
- 2
- 21
- 60
0
votes
1 answer
Starting Apache Nifi with Apache Airflow - how to generate a flow file
I have a setup of Apache Nifi for my ETL pipeline and want to start (and later monitor) a specific processor with Apache Airflow.
I see two ways of achieving this from within an airflow DAG:
Generate a flow file from scratch and insert it into a…
Cribber
- 2,513
- 2
- 21
- 60
0
votes
2 answers
Generating Single Flow file for loading it into S3
I have a Nifi Flow, which fetches a data from RDS tables and load into S3 as flat files, now i need to generate another file which will be having the name of the file that I am loading into S3 bucket, this needs to be a separate flow;
example: if…
Vasanth Kumar
- 7
- 1
0
votes
1 answer
How to create multiple flow files from one incoming flow files in nifi using ExecuteScript with python
Running locally, this works exactly how I want it to (has one incoming flow file with many different codes in position 7-10 and outputs 1 file per unique code) For example if record 1-5 has 1234 in positions 7-10, and record 6 has 2345 in position…
andym
- 1
- 1
0
votes
1 answer
Apache NiFi Distributed Performance
I have a file which is approximately 40GB in size, my nifi cluster runs on one node, and takes around 60 minutes to decompress. If I have this Processor over 2 or 3 nodes does the processor distribute its work for the decompress time to go down to…
Agus G
- 1
0
votes
1 answer
Deleting flowfiles from apache nifi
Below is the flow that I'm working on.
InvokeHTTP-->ExecuteScript-->ExecuteSQL
From invoke HTTP, I'm getting two flow files one with request and another with a response. I've configured ExecuteSQL with Event-Driven scheduling. Since there are two…
Ramu Chowdam
- 33
- 2
- 11
0
votes
2 answers
NIFI EXECUTESCRIPT Processor failing - No MODULE FOUND
I am trying to import modules into executescript processor in nifi.
As suggested , I. am giving full path into the modules directory.
example:
Module Directory: /var/lib/nifi/Levenshtein --> which contains necessary files for the…
devanshmbi
- 3
- 3
0
votes
1 answer
Nifi Custom processor showing error "the local variable flowfile cannot be assigned"
I am trying to send the original flowfile as input to next processor, but ending up getting the error, I am very new to Nifi and also some experience in Java.
public void onTrigger(ProessorContext context, ProcessSessionsession) throws…
Kumar Prvn
- 31
- 1
- 7
0
votes
0 answers
Apache Nifi read flowfile content
I would like to access the content of a flow file that should contain binary data but when I execute this code:
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
FlowFile ff =…
magol02
- 1
- 1