0

We have many DAGs scheduled to run daily using Airflow. Dependencies has been enabled using ExternalTaskSensor, TriggerDagRunOperator and custom operators

Sample:

Task 1 in DAG A are dependent on task 2 in DAG B
Task 3 in DAG A are dependent on task 4 in DAG C
Task 5 in DAG A are dependent on task 6 in DAG D
...

Task 2 in DAG B are dependent on task 7 in DAG E
Task 4 in DAG B are dependent on task 8 in DAG F ...

While checking Task Instance details in UI, only downstream_task_ids and upstream_task_ids belonging to the same dag are displayed.

How can we see the full lineage of a single task across multiple DAGs to the last available level?

1 Answers1

1

Airflow does not currently (v 1.8.1) have a mechanism for viewing cross-dag dependencies.

At this time if you need a visualization of relationships between tasks, they have to be in the same dag. Potentially a view in a custom plugin could show these dependencies, but the stock UI does not do this.

jhnclvr
  • 9,137
  • 5
  • 50
  • 55