2

I have a dag that I want to run multiple times say 30. But airflow can parallelly execute 16 dag runs at a time. Suppose one dag run takes longer time to execute , then among the 14 dag runs waiting , I want to run 10th dag run first. Is there any way in airflow to achieve this? I know there are ways through which we can set priority weights for different dags and tasks. But I want to assign priority weights to dag runs.

In short, I am looking for a way to set priorities for the multiple dag runs of same dag ?

Thanks.

ansh
  • 21
  • 1

1 Answers1

0

You can you use pools, and limit number of parallel runs within the pool to 16, in that case you make sure that you prioritize the 16 runs from same dag.

Atef Arfaoui
  • 53
  • 1
  • 7
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community May 04 '22 at 10:36