Airflow dag executing middle tasks after completing stop_dag task, how to stop other tasks want to dependency even though parent task did not ran

Question

Here is the image after stop execution step, highlighted steps are running without any dependency. please help to stop other task after stop_dag task run completed.

Answer 1

Not completely sure I understand you correctly but you want that the red marked tasks don't run in case stop_dag is successful? This can be accomplished using trigger rules and one extra task.

I tried to recreate your setup with EmptyOperators:

Then I add a set of dependencies and an additional task that fails if stop_task is successful due to the trigger rule all_failed:

fail_if_stop_successful = EmptyOperator(
    task_id="fail_if_stop_successful",
    trigger_rule="all_failed"
)
stop_task >> fail_if_stop_successful >> [t3_1, t3_2, t3_3]

This creates the following pattern:

Full DAG:

from airflow.decorators import dag
from airflow.operators.empty import EmptyOperator
from pendulum import datetime

@dag(
    start_date=datetime(2023,1,1),
    schedule=None,
    catchup=False
)
def test_dag():

    start = EmptyOperator(task_id="start")

    t1 = EmptyOperator(task_id="t1")

    stop_task = EmptyOperator(task_id="stop_task")
    
    t2_1 = EmptyOperator(task_id="t2_1")
    t2_2 = EmptyOperator(task_id="t2_2")
    t2_3 = EmptyOperator(task_id="t2_3")

    t3_1 = EmptyOperator(task_id="t3_1")
    t3_2 = EmptyOperator(task_id="t3_2")
    t3_3 = EmptyOperator(task_id="t3_3")

    t4 = EmptyOperator(task_id="t4")

    end = EmptyOperator(task_id="end")

    # rebuilding the dependencies from your drawing 
    start >> t1 >> [t2_1, t2_2, t2_3]
    start >> stop_task
    t2_1 >> t3_1
    t2_2 >> t3_2
    t2_3 >> t3_3
    [t3_1, t3_2, t3_3] >> t4 >> end

    # making a successful run of stop_task stop t3_1, t3_2, t3_3
    fail_if_stop_successful = EmptyOperator(
        task_id="fail_if_stop_successful", trigger_rule="all_failed"
        )

    stop_task >> fail_if_stop_successful >> [t3_1, t3_2, t3_3]

test_dag()