I am evaluating Talend and I have little experience using the tool. My question is regarding audit. Does Talend offer a way to run with an audit option that extracts the source data, transforms it and then compares the transformed data to the existing target data? I would like a report that tells me which records would have triggered an update if the audit option had not prevented it.
I know that ETL's are triggered on change but I would want to run this audit without regard to changed timestamps. That is, a query would specify what source records to audit.
I have done a search of the talend documentation but nothing jumped out at me.
Thank you.
EDIT
Looking outside of Talend, I found where Tim Mitchell wrote:
While I have not yet found an auditing framework with enough customization to suit me, I have learned that it is possible to audit ETL processes in a customized way without reinventing the wheel every time. I keep on hand a common (but never set in stone) set of table definitions and supporting logic to shorten the path to the following auditing objectives:
- Simple aggregate auditing of row counts, financial data, and other key metrics
- Documentation templates for source-to-target mappings and transformation logic
- Lightweight auditing of batch-level changes
- Full auditing of every change made in the transformation process
So I guess that auditing is on me.
