0

I am currently learning how to use Azure Data Factory. I try to do a data ingestion and transformation that I did in a Python script. Now I try to do the same on Data Factory as it is supposed to be easier.

I have a zipped folder. It contains Tar files. Each file contains zipped cvs file. By ingesting csv file directly to a blob it would be easy of course but if I have to automate the reception of such a zipped folder how would I be able to load the csv in a database if I even can't figure out how to unzip, detar and unzip again? In this cas I use climate data from a real case study... I you have any idea how to handle this problem I would appreciate! Thanks in advance!!!

user11472129
  • 1

1 Answers1

0

Based on the official document, adf blob storage dataset only supports below compression type:

enter image description here

Tar format is not supported by adf naturally. However,you could follow the solution which is mentioned in the same document, using Azure Function to extract the contents of a tar file or build this functionality using a custom dotnet activity. Considering your slightly complicated situation, I suggest you using custom activity.

Jay Gong
  • 23,163
  • 2
  • 27
  • 32