0

I'm going through an unusual situation with Data Factory, I'm copying the fields using the COPY component where the source is a SQL SERVER base and the Destination is a table in TABLE DELTA LAKE format. From the image I put together here, the number of lines read is 4636 but the destination is 3785. Did the process run without any errors? Any idea what could be going on?

The most bizarre thing is that there are other tables that normally copy data from the same SQL source database, only this table has this row inconsistency

enter image description here

user3661384
  • 524
  • 7
  • 18

1 Answers1

1

Data Factory does not copy all rows to SINK destination in DELTA LAKE

This can be done because of some entries could be omitted from the copy because of changes in the schema between the source and destination tables.

To Identify why some rows are not getting copied in destination in DELTA LAKE you can use the logging option in copy activity settings.

enter image description here

It will store the logs of skipped rows in the specified destination blob storage or ADLS.

Sample logs output:

enter image description here

from here you can understand the reason for every skipped row.

Pratik Lad
  • 4,343
  • 2
  • 3
  • 11