13

I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.

I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS. This SO post suggests using the GUI to "clear" failed task:

How to restart a failed task on Airflow

This approach works if you have a handful number of failed tasks.

I am wondering if we can bypass the GUI and do it problematically, through command line something like:

airflow_clear_failed_tasks dag_id execution_data
motam79
  • 3,542
  • 5
  • 34
  • 60

2 Answers2

10

The command to clear only failed tasks was updated. It is now (Airflow 2.0 as of March 2021):

airflow tasks clear [-s START_DATE] [-e END_DATE] --only-failed dag_id
pgzmnk
  • 202
  • 2
  • 5
  • 2
    use the option as --only_failed or -f – Shams Apr 14 '21 at 08:49
  • 4
    According to [the docs](https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html), date format when using this command is `YYYY-MM-DD` – swimmer Jun 10 '21 at 07:54
9

Use the following command to clear only failed tasks:

airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id

Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear

kaxil
  • 17,706
  • 2
  • 59
  • 78