1

I have a situation where I am using Data Flows to ingest data from Blob Storage. The file format is CSV, however some files are comma delimited and some are pipe delimited.

It is possible to parameterise the delimiter of the dataset itself in ADF.

enter image description here

However in the Dataflow activity the Source Settings do not provide the option to either dataset parameters or change the delimiter.

enter image description here

It's not even possible to parameterise the choice of dataset. This means that I have to create a new Dataflow for each delimiter which is not practical.

Adil Khan
  • 28
  • 5
  • Hi Adil Khan, If my answer is helpful for you, you can mark it as answer( click on the check mark beside the answer to toggle it from greyed out to filled in.). This can be beneficial to other community members. Thank you. – Leon Yue Apr 29 '20 at 01:05
  • Hello, Adil, Khan, have you tried this? If you still have questions, please let me know, thanks. – Leon Yue Apr 30 '20 at 02:04

1 Answers1

3

I tried this and it works well.

Source dataset parameter setting, add parameter ColDelimiter:

enter image description here

Create a pipeline parameter ColDelimiter:

enter image description here

Then click the Data Flow, we can set the source parameter:

enter image description here

When run the pipeline, we could set the column delimiter from paramter: enter image description here

Hope this helps.

Leon Yue
  • 15,693
  • 1
  • 11
  • 23