3

Have a dag like this:

import os
from datetime import timedelta

from xxx import on_failure_opsgenie
 
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
 
DAG_ID = os.path.basename(__file__).replace(".py", "")

DEFAULT_ARGS = {
    "owner": "airflow",
    "depends_on_past": False,
    "email": ["airflow@example.com"],
    "email_on_failure": False,
    "email_on_retry": False,
}
 
 
def kaboom(*args, **kwargs):
    print("goodbye cruel world")
    print(args)
    print(kwargs)
    assert 1 == 2
 
 
with DAG(
    dag_id=DAG_ID,
    default_args=DEFAULT_ARGS,
    description="Print contents of airflow.cfg to logs",
    dagrun_timeout=timedelta(hours=2),
    start_date=days_ago(1),
    schedule_interval=None,
    on_failure_callback=on_failure_opsgenie,
) as dag:
    get_airflow_cfg_operator = PythonOperator(task_id="gonna_explode", python_callable=kaboom)

The DAG fails as expected, purposefully. However, on_failure_opsgenie is not doing what it should; how do I get the logs or debug a failed on-failure-callback in AWS MWAA?

Tommy
  • 12,588
  • 14
  • 59
  • 110
  • does `on_failure_opsgenie` take a context argument? see https://stackoverflow.com/questions/65967548/airflow-on-failure-callback – dovregubben Dec 24 '21 at 10:14

1 Answers1

1

The first step would be to look at the definition of the on_failure_opsgenie. If memory serves, exceptions in the callback should still print to the task log. However, if they don't, you can create a PythonOperator and pass python_callable=on_failure_opsgenie. Every time the task runs, it will try to execute your callback, and any exceptions should definitely end up in the log. You can then go from there.

I find that it's also helpful to keep these callbacks as simple as possible, so that when they fail, it is more or less obvious where the issue is.

Dommondke
  • 307
  • 1
  • 8