Get failure reason in Apache NiFi

Question

I want to log the reason of a failure in some NiFi processor. but i saw that it could have multiple reasons.

Example for SplitAvroJson processor: "If a FlowFile fails processing for any reason (for example, the FlowFile is not valid Avro), it will be routed to this relationship"

Is there any possibility to get the exact reason of my failure in order to save it in for example in an put file?.

Thank you in advance.

score 3 · Answer 1 · answered Apr 27 '16 at 14:18

3

Most processors will log the error to nifi-app.log. For the example of SplitAvro it does that here:

https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/src/main/java/org/apache/nifi/processors/avro/SplitAvro.java#L206

This error message would also be visible in the NiFi UI as a bulletin on the SplitAvro processor.

Does that achieve what you are looking for?

answered Apr 27 '16 at 14:18

Bryan Bende

18,320
1
28
39

Thabk you for your answer, but if it's possible i want to get the error message like an attribue in order to save it with other attribue (example filename) in hbase table. – Amine CHERIFI Apr 27 '16 at 15:16
1

I'm not sure there is anything that does exactly what you are describing, but another option might be to implement a custom ReportingTask. A ReportingTask has access to the BulletinRepository which is where all of the bulletins for the flow are stored, and you could send them somewhere outside NiFi. The only limitation is that Bulletins are disconnected from the FlowFile that generated the bulletin, so all you will know is that there was a specific error message at a given time from a given component, but you won't have access to the FlowFile that produced it. – Bryan Bende Apr 28 '16 at 18:40
When ValidateRecord rejects a records , it doesn't tell which column caused the issue . How can we get such information – Aviral Kumar Aug 21 '20 at 02:14

Swati Sood · Answer 2 · 2016-05-26T17:24:33.447

i had a similar question, i think this post might help u.

You have a couple of different options here to consume the bulletin messages using the rest api:

1) http[s]://{host}:{port}/nifi-api/controller/process-groups/{process-group-id}/status?recursive=true

This request will get the status (including bulletins) of all components under the specified Process Group. You can use the alias 'root' for the root level Process Group. The recursive flag will indicate whether or not to return just the children of that Process Group or all descendant components.

2) http[s]://{host}:{port}/nifi-api/controller/status

This request will get the status (including bulletins) of the Controller level components. This includes any reported bulletins from Controller Services, Reporting Tasks, and the NiFi Framework itself (clustering messages, etc).

3) http[s]://{host}:{port}/nifi-api/controller/bulletin-board?limit=n&sourceId={id}&message={str}

This request will access all bulletins and supports filtering based components, message and limiting the number of bulletins returned.

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/low-quality-posts/12484743) — MLavoie, May 26 '16 at 16:25

Get failure reason in Apache NiFi

2 Answers2