1

How do you fill ExternalTaskSensor parameter external_dag_id?

  1. Hardcode. It's very simple but hard to mantain DAG, if external DAG's dag_id changed you must keep in mind to change external_dag_id also. Even some misspell that is unware.
  2. or import that dag and use external_dag.dag_id. This solve problem above. But it may issue cyclic dependency and some global variables problem. These problems are more unware than Hardcode solution.

Do anyone have some best practice abount this in production? Any advises are welcome.

MoreFreeze
  • 2,856
  • 3
  • 24
  • 34

1 Answers1

1

I recommend using the second solution.

If there have global variables, you should move them into Airflow Variables. And to solve the cyclic dependency problem, you need to reconstruct the DAG. It depends on Pipelines.

Junyoung Park
  • 66
  • 4