0

I wanted to know how to get list of in-built pipeline arguments in Data Fusion Pipeline ?

I am not able to find them anywhere on the documentation as well as on the internet.

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
SUDHIR GARG
  • 75
  • 9
  • What do you mean with in-built arguments? Can you provide an example? – rmesteves Aug 20 '20 at 11:19
  • Built-in arguments which are already there with the pipeline by default like one in know 'logicalStartTime'. – SUDHIR GARG Aug 20 '20 at 17:37
  • logicalStartTime is the only argument added by the system. Even for that, I would recommend using the macro (https://docs.cdap.io/cdap/6.2.1/en/developer-manual/pipelines/creating-pipelines.html#H4916) instead of accessing it directly, as it has additional features for formatting the time or doing simple math on it. – Albert Shau Aug 20 '20 at 18:31
  • @SUDHIRGARG does Albert comment answers your question? – rmesteves Aug 25 '20 at 14:54
  • Thanks for the answer Albert, That answers my question – SUDHIR GARG Aug 27 '20 at 09:18

1 Answers1

1

As discussed, logicalStartTime is the only argument added by the system.

To get some parameter even those added by the system its recommended using macros that allow placeholders for properties that are unknown at configure time but are known and provided at runtime. For logicalStartTime we have the pre-defined macro named Logical Start Time Function that can be used like below:

${logicalStartTime([timeFormat[,offset [,timezone])}

Where:

  • timeFormat: Time format string, in the format of a Java SimpleDateFormat
  • offset: An offset of time before the actual logical start time
  • timezone: Timezone to be used for the logical start time

The documentations shows us an example:

For example, suppose the logical start time of a pipeline run is 2016-01-01T00:00:00 and this macro is provided:

${logicalStartTime(yyyy-MM-dd'T'HH-mm-ss,1d-4h+30m)}

The format is yyyy-MM-dd'T'HH-mm-ss and the offset is 1d-4h+30m before the logical start time. This means the macro will be replaced with 2015-12-31T03:30:00, since the offset translates to 20.5 hours. The entire macro evaluates to 20.5 hours before midnight of January 1, 2016.

rmesteves
  • 3,870
  • 7
  • 23