0

I have created a pipeline which queries out a database for new records, I do this using the Last Modified TimeStamp field. In my query I want to retrieve only records modified since the last time the pipeline was run.

How can I do this? I'm new to SnapLogic but I know I need to essentually store the runtime as some sort of savable variable which can be used within expressions in the pipeline itself. I also need to have a fallback incase the variable doesn't exist (i.e. when its the first time its been run).

InvalidSyntax
  • 9,131
  • 20
  • 80
  • 127

1 Answers1

0

You can save the timestamp in a file in SLDB against the pipeline name.

So, when your pipeline starts, read the timestamp against the pipeline name from the file (you can put 0 by default if there is not an entry for it in the file) then do whatever processing you want to do. After successful execution of your processing, you can save the pipe.startTime() against the pipeline name in the above-mentioned file; you have to replace the previous entry in all subsequent executions.

Bilesh Ganguly
  • 3,792
  • 3
  • 36
  • 58