1

I have 20 bash scripts. Some run every min, every day and some on every hour using cron. Now I need to migrate in airflow. For this, as per the airflow concept, I need to create 20 more files (DAG file).

Does airflow provide away to create generic dag template which can execute all the bash scripts on given schedule time with different dag id?

I got a reference - Airflow dynamic DAG and Task Ids But I am in doubt, is it the right way or not?

lucy
  • 4,136
  • 5
  • 30
  • 47

1 Answers1

0

You can create 3 DAGS:

  • DAG scheduled every minute
  • hourly scheduled DAG
  • daily scheduled DAG.

Under these DAGS create tasks to execute your corresponding scripts.

esugei
  • 181
  • 1
  • 1
  • 8
  • I have 20 bash scripts, then I need to create 20 dag files. How I can maintain with a single file? – lucy Jan 07 '20 at 13:44