3

I have followed the documentation and previous stackoverflow links eg : Airflow sla_miss_callback function not triggering Still I am not able to trigger sla_miss_callback for my case.

I have a top level DAG(DAG X1) that has scheduler interval of : 0 18 * * * -> run everyday at 6 pm. This calls sub DAG(DAG Y1) using TriggerDagRunOperator. Sub DAG Y1 do not have schedule interval attached so I cannot apply SLA to it because of this piece of code - Airflow Code

So I have attached SLA to my top level DAG task exactly the same way as this - https://airflow.apache.org/docs/apache-airflow/2.3.1/concepts/tasks.html#slas

@dag(
    schedule_interval="0 18 * * *",
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    sla_miss_callback=sla_callback,
    default_args={'sla': timedelta(seconds=60)
}

def sla_callback(dag, task_list, blocking_task_list, slas, blocking_tis):
    print(
        "The callback arguments are: ",
        {
            "dag": dag,
            "task_list": task_list,
            "blocking_task_list": blocking_task_list,
            "slas": slas,
            "blocking_tis": blocking_tis,
        },
    )

In my DAG Processor log I do see this log {{processor.py:377}} INFO - Running SLA Checks for DAG X1, Coming from this part of the Airflow Code but after that I dont see any log related to SLA and SLA Callback function is not getting kicked off. My DAG run time is more than hour long.

Mayank Kataruka
  • 395
  • 1
  • 2
  • 11
  • What is the configuration for logging for the DagProcessor? – Priyadarshan Mohanty Jun 02 '22 at 17:19
  • We are facing the same issue here, we found that our `DAGFileProcessorProcess` is being killed by the processor. We think this might be the case, because the callback is made inside of the processor. Will keep you updated. – Thiago Baldim Aug 03 '22 at 23:03
  • We found the logs of the slas on `/airflow/logs/scheduler/latest/workflow/dags` that for some reason doesn't go to the logs in the web. That is where we could find the details why our `sla_callbacks` are not working. – Thiago Baldim Aug 04 '22 at 00:19

0 Answers0