0

I want to fail the task if task duration exceeds a certain time for which I configured execution_timeout in every task of the dag. However, the task is failing, but I am not getting email notification of it. Does anyone know the reason? Any help would be appreciated.

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime.today() - timedelta(days = 2),
    'email':['email@company.com'],
    'email_on_failure': True
}

[2022-10-27 03:06:42,587] {logging_mixin.py:112} INFO - [2022-10-27 03:06:42,585] {timeout.py:42} ERROR - Process timed out, PID: 15847
[2022-10-27 03:06:42,588] {bash_operator.py:140} INFO - Sending SIGTERM signal to bash process group
[2022-10-27 03:06:42,602] {taskinstance.py:1145} ERROR - Timeout, PID: 15847
Traceback (most recent call last):
  File "/home/airflow_user/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 978, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/home/airflow_user/.local/lib/python3.7/site-packages/airflow/operators/bash_operator.py", line 124, in execute
    for line in iter(self.sub_process.stdout.readline, b''):
  File "/home/airflow_user/.local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
    raise AirflowTaskTimeout(self.error_message)
airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 15847

I can see the task turning to red. But no email is sent even with the email_on_failure config.

Ken White
  • 123,280
  • 14
  • 225
  • 444
kavya
  • 75
  • 1
  • 10
  • how do you configure your email in airflow.cfg? – Toàn Nguyễn Văn Oct 27 '22 at 03:25
  • ``` [email] email_backend = airflow.utils.email.send_email_smtp ``` – kavya Oct 27 '22 at 03:30
  • What about "smtp_host", "smtp_user", "smtp_password", "smtp_mail_from" in your airflow.cfg? – Toàn Nguyễn Văn Oct 27 '22 at 03:34
  • # If you want airflow to send emails on retries, failure, and you want to use # the airflow.utils.email.send_email_smtp function, you have to configure an # smtp server here smtp_host = localhost smtp_starttls = True smtp_ssl = False # Example: smtp_user = airflow # smtp_user = # Example: smtp_password = airflow # smtp_password = smtp_port = 25 smtp_mail_from = airflow@quotient.co – kavya Oct 27 '22 at 03:37

1 Answers1

0

You have not configured your smtp in airflow.cfg, you need configure it:

[smtp]
smtp_host = #your smtp host
smtp_starttls = True
smtp_ssl = False
# Example: smtp_user = airflow
smtp_user = # your email to send
# Example: smtp_password = airflow
smtp_password = # App password
smtp_port = # your smtp port
smtp_mail_from = # your email to send
smtp_timeout = 30
smtp_retry_limit = 5

Example, if you are using gmail:

[smtp]
smtp_host = smtp.gmail.com
smtp_starttls = True
smtp_ssl = False
# Example: smtp_user = airflow
smtp_user = example@gmail.com
# Example: smtp_password = airflow
smtp_password = #app password
smtp_port = 25
smtp_mail_from = example@gmail.com
smtp_timeout = 30
smtp_retry_limit = 5

Note: you can take your app password in here: https://support.google.com/mail/answer/185833?hl=en

Gmail port can be 25, 465 or 587

Toàn Nguyễn Văn
  • 151
  • 5