0

For Incremental load we will be using QueryDatabaseTable processor which extracts data incrementally from one table. For writing sql query which extracts data from multiple tables we are using ExecuteSQL processor.

How can we extract incremental load for a join query??

Anil Krishna Parvathaneni
  • 43
  • 1
  • 5

1 Answers1

0

If I understand what you're trying to do, in NiFi that's a Lookup pattern so you'd likely use LookupRecord with a DatabaseRecordLookupService. Each one of those would "join" the incremental load table with the rows from the table specified by the DatabaseRecordLookupService. For multiple joins you'd have a LookupRecord with corresponding DatabaseRecordLookupService for each of them.

mattyb
  • 11,693
  • 15
  • 20
  • Could you please provide any sample screenshot of configurations – Anil Krishna Parvathaneni Dec 14 '21 at 14:05
  • There are examples of LookupRecord at https://community.cloudera.com/t5/Community-Articles/Data-flow-enrichment-with-NiFi-part-1-LookupRecord-processor/ta-p/246940 but not with DatabaseRecordLookupService – mattyb Feb 23 '22 at 23:57