0

Is there a way to use NiFi cron driven scheduler or other ways to run the process for a specific time a day, or to stop for a brief moment.

For example: every day I want the scheduler to stop between 5am-6am & 5pm-6pm, and the rest of the time is continuous.

xlm
  • 6,854
  • 14
  • 53
  • 55
John
  • 1
  • 1
  • NiFi cron scheduler use the same syntax as all Linux crons. You could easily google some cron expression generator (for example https://www.freeformatter.com/cron-expression-generator-quartz.html) and select only minutes, hours, days when you want execute your processor. – 32cupo Jun 04 '20 at 09:50

1 Answers1

0

The CRON scheduler allows for specifically-timed discrete invocations, rather than "on during this period and off during this period". I've crafted a CRON schedule expression which sort of meets your request, but I have a different suggestion instead (CRON scheduling is also not an encouraged approach; it was implemented in NiFi for legacy reasons but isn't really actively developed or recommended).

The CRON expression

0-59 0-4,6-16,18-23 * * *

This expression means "run (the processor) at minute 0, 1, ..., 59 of hours 0-4, 6-16, 18-23 (midnight through 4:59am, 6:00am through 4:59pm, and 6:00pm through 11:59pm) everyday." Your processor will run 1320 times each day (60 times per hour for 22 hours). However, this means it will run once per minute, which may or may not be frequent enough for your needs.

Better suggestion

My actual solution would be to enable the processor to run on the actual Run Schedule you want (every 1 sec, every 3 min, etc.) and use a script to invoke the HTTP API (or Python API) to enable/disable the processor at the specified times. You can even use the CRON scheduling capability of your operating system to do so.

Example:

  • In NiFi, Processor X is enabled and running.
  • At 0500 daily, a script is run (via CRON or otherwise), which invokes curl -X PUT http://my.nifi.service:8443/nifi-api/processors/... and updates the state to STOPPED.
  • At 0600 daily, a script is run which updates the state to STARTED again.
  • Repeat at 1700 and 1800.

You can parameterize the script to accept an argument determining start/stop to reduce duplication of code. I think this is a more robust solution to your problem.

Other resources:

You can also make any specific request via the GUI and use the browser's developer tools to record the exact API invocation to capture the processor UUID, JSON values, etc.

Andy
  • 13,916
  • 1
  • 36
  • 78