0

I want to copy the file from Source to target container but only when the Source file is new (latest file is placed in source). I am not sure how to proceed this and not sure about the syntax to check the source file greater than target. Should i have to use two get metadata activity to check source and target last modified date and use if condition. i tried few ways but it didn't work. Any help will be handy

enter image description here

syntax i used for the condition is giving me the error

@if(greaterOrEquals(ticks(activity('Get Metadata_File').output.lastModified),activity('Get Metadata_File2')),True,False)

error message

The function 'greaterOrEquals' expects all of its parameters to be either integer or decimal numbers. Found invalid parameter types: 'Object'

user2703679
  • 35
  • 2
  • 10
  • Also refer to one my answers [here](https://stackoverflow.com/a/68560714/15969417) and [here](https://stackoverflow.com/a/69551445/15969417) – KarthikBhyresh-MT Dec 15 '21 at 08:03
  • Thanks for the reply but i tried it out but the problem is folder path and directory are static, i have three different environments and how would i change the variable in the folder source path from (DEV/TEST/PROD) dynamically. Thats why i though of using the condition if container file is older than source file then copy or else don't copy but syntax is not working properly and i tried to change the syntax to the following @if(less(ticks(activity('Get Metadata_File').output.lastModified),ticks(adddays(utcnow(),0))),True,False) this works but not expected behaviour – user2703679 Dec 16 '21 at 00:24
  • 1
    I figured out where the syntax was going wrong, Fixed the syntax and got the right results @if(less(ticks(activity('Get Metadata_File').output.lastModified),ticks(activity('Get Metadata_File2').output.lastmodified)),True,False) – user2703679 Dec 16 '21 at 01:21

1 Answers1

1

You can try one of the Pipeline Templates that ADF offers.

enter image description here

enter image description here

Use this template to copy new and changed files only by using LastModifiedDate. This template first selects the new and changed files only by their attributes "LastModifiedDate", and then copies them from the data source store to the data destination store. You can also go to "Copy Data Tool" to get the pipeline for the same scenario with more connectors.

View documentation

OR...

You can use Storage Event Triggers to trigger the pipeline with copy activity to copy when each new file is written to storage.

Follow detailed example here: Create a trigger that runs a pipeline in response to a storage event

KarthikBhyresh-MT
  • 4,560
  • 2
  • 5
  • 12