How to get reason for failure using slack in airflow2.0

Question

How to get the reason for the failure of an operator, without going into logs. As I want to post the reason as a notification through slack?

Thanks, Xi

score 1 · Answer 1 · answered Apr 05 '22 at 22:36

1

I can think of one way of doing this as below.

set error notifications -> https://www.astronomer.io/guides/error-notifications-in-airflow/
Also create a slack email alias for DM https://slack.com/help/articles/206819278-Send-emails-to-Slack
Other way is using the Slack API from airflow : https://medium.com/datareply/integrating-slack-alerts-in-airflow-c9dcd155105

Check the above for SlackAPIPostOperator

answered Apr 05 '22 at 22:36

hopeIsTheonlyWeapon

557
6
19

Slack notifications are working for me. I need the exact reason why a task has failed. – Xi12 Apr 05 '22 at 22:38
@Xi12 then check the first option where you integrate slack using email notifications. Where the task errors will go to the email alias which in turn will go to slack DM. Hope that helps. – hopeIsTheonlyWeapon Apr 05 '22 at 22:40
I saw it, but what is the function that would give me exact reason for failure? – Xi12 Apr 05 '22 at 22:42
Check this out . https://stackoverflow.com/questions/44573923/how-do-i-setup-airflows-email-configuration-to-send-an-email-on-errors Also make sure your airflow configuration is set up properly with respect to the smtp params – hopeIsTheonlyWeapon Apr 05 '22 at 22:45

score 1 · Answer 2 · answered Apr 06 '22 at 00:25

exception=context.get('exception')is the function which will give exact reason for failure

Example of on_failure_callback using slack:

 step_checker = EmrStepSensor(task_id='watch_step',
                 job_flow_id="{{ task_instance.xcom_pull('create_job_flow', 
                 key='return_value') }}",
        step_id="{{task_instance.xcom_pull(task_ids='add_steps',key='return_value')[0] }}",
        aws_conn_id='aws_default',
        on_failure_callback=task_fail_slack_alert,)    
    

def task_fail_slack_alert(context):
        SLACK_CONN_ID = 'slack'
        slack_webhook_token = BaseHook.get_connection(SLACK_CONN_ID).password
        slack_msg = """
                :red_circle: Task Failed. 
                *Task*: {task}  
                *Dag*: {dag} 
                *Execution Time*: {exec_date}  
                *Log Url*: {log_url} 
                *Error*:{exception}
                """.format(
                task=context.get('task_instance').task_id,
                dag=context.get('task_instance').dag_id,
                exec_date=context.get('execution_date'),
                log_url=context.get('task_instance').log_url,
                exception=context.get('exception') 
               
            )
        failed_alert = SlackWebhookOperator(
            task_id='slack_test',
            http_conn_id='slack',
            webhook_token=slack_webhook_token,
            message=slack_msg,
            username='airflow',
            dag=dag)
        return failed_alert.execute(context=context)

How to get reason for failure using slack in airflow2.0

2 Answers2