0

I have a job which runs everyday and based on column 'modifyts' pulls the records from db2 as delta: modifyts>current_date-1.

Recently we found out that some of the data is getting missed and not loaded to our netezza target table.

Is there a way we can get the missing data?

As of now, we are planning to load for past 3 days with condition modifyts>current_date-3 but it could cause data error in other process. Is there an efficient way to achieve this?

Any suggestions will greatly help. thanks in advance

Amin
  • 9
  • 1
  • 4
  • What do you mean some of your data is not loaded? Do you use any reject methods? Have you identified any pattern for the missing data? I would first identify which data is missing and observe the ETL job a little bit. – VSK Jul 04 '21 at 12:37
  • On source end its a db2 CDC process so there may be a chance where the rows in db2 table would have been updated after our etl extract job due to latency hence missing that record forever. There's nothing wrong with our etl job. – Amin Jul 05 '21 at 18:01

0 Answers0