1

Hi I am trying to sink csv file to databse using ADF and Data Flow CSV file structure as below r1c1,r1c2,r1c3,r1c4

r2c1,r2c2,r2c3,r2c4

r3c1,r3c2,r3c3,r3c4

r4c1,r4c2,r4c3,r4c4,r4c5,r4c6

r5c1,r5c2,r5c3,r5c4,r5c5,r5c6

so 2 schemas one with 4 columns and second with 6 columns. My goal is to copy first 3 rows in one table and last 2 in second table. Please guide me to acheive this in ADF.

Rahulkumar Patil
  • 11
  • 1

1 Answers1

0

Your useCase is typical for ConditionalSplit transformation. here is a simple demo that i created to demonstrate your usecase.

  1. add 2 schemas as datasets in ADF
  2. create a pipeline with Data Flow activity
  3. in DataFlow : add your souce (CSV data) , i put Null values in the first 3 rows in columns 5 and 6 as you requested.
  4. in Conditional split , i added this condition : "isNull(Column_5) && isNull(Column_6)" - see screenshot below.
  5. Rows that matches the condition will be saved in the matching sink.

Data Flow Activities: enter image description here

Conditional Split Activity: enter image description here

Rows 1 - 3 output : enter image description here

Rows 5 to 6 output: enter image description here

Read more about it here : https://learn.microsoft.com/en-us/azure/data-factory/data-flow-conditional-split

Sally Dabbah
  • 411
  • 2
  • 8