I'm trying to dynamically get the name of the parquet file that is created in the copy, but I can't find it in the ouput. They could help me identify how to get the name, to use in a next activity.
Asked
Active
Viewed 307 times
0
-
You could use a GetMetadata activity after the Copy to get the file name from the Sink folder location. – Joel Cochran Dec 23 '21 at 17:29
1 Answers
0
You can follow any of the below processes.
Process1:
You can generate the file name to generate the parquet file with the given file name in copy data activity and use the same file name in later activities.
While creating the sink dataset, create a dataset parameter for the file name and provide the file name in copy data sink settings.
Parameterize the dataset:
Copy activity sink:
You can use the same name in later activities or create a variable to store this name and use it.
Process2:
You can get the latest file generated in the folder using the Get Metadata activity.
Refer to this SO thread for repro details.
NiharikaMoola-MT
- 4,700
- 1
- 3
- 15
-
Thank you very much @NiharikaMoola-MT, with process 2 you can get the original name of the parquet file that is generated after making a copy. – Eddynson Vega Dec 24 '21 at 17:41