0

I have a Airflow DAG with around 10 tasks in it.

So now, I have another two tasks to execute only after execution of all these 10 tasks earlier.

Can you help me with this case.

Shanmukh S
  • 81
  • 1
  • 9
  • If you are using Airflow 2.0.0 or greater, [this answer](https://stackoverflow.com/a/68367685/10569220) may help you. Specially if you want to set dependencies between two groups of tasks. – NicoE Aug 30 '21 at 15:58

1 Answers1

2

You can specify in your dag all the dependencies between your tasks with the right-bitshift operator.

Example for 3 tasks (make it up to your 10 tasks) :

[task1, task2, task3] >> final_task

See this post for more info.

Val Berthe
  • 1,899
  • 1
  • 18
  • 33
  • If I add this piece of code , It would execute final_task three times. Like after execution of task1 , after execution of task2 and after execution of task3. But thats not my requirement. My requirement is , final_task should execute after complete of task1 , task2 and task3 – Shanmukh S Aug 30 '21 at 14:18
  • Please check Airflow documentation [here](http://airflow.apache.org/docs/apache-airflow/1.10.5/concepts.html#bitshift-composition). – Val Berthe Aug 30 '21 at 15:19
  • @KoradaGanesh _If I add this piece of code , It would execute final_task three times. Like after execution of task1 , after execution of task2 and after execution of task3_ I thought the same thing because when I do this, final_task shows up 3 different times in the tree view. However, if you look on the Graph tab, the DAG is in the desired format. Each final_task in the tree view is actually the same task. If you clear one, you clear them all and if you mark one as failed, they're all marked as failed. – Joe Jan 18 '22 at 13:57