0

There is this git repository publicly available. It's being refreshed daily. There are several csv files with the structure like "DA-01-12-2022", "DA-02-12-2022", "DA-03-12-2022" and so on. The date is in the file name. It's also in the githublink, so I can copy one file without problem but since there are many CSV files in the git folder how can I use Synapse pipelines to copy all the files in the git repository to a storage in azure. I feel like I have to use loops but how can I tell it to use the date?

Thanks and best regards!

LightningStack575
  • 29
  • 5
  • but how can I tell it to use the date?--**Could you explain this in detail? Do you want to copy all files or based on the date?** – Aswin Dec 10 '22 at 03:40

1 Answers1

0

You can use a copy activity to load all csv and store in one parquet files

enter image description here

You can also use ITER activity to make loop with date

nicolas
  • 26
  • 1