0

Getting this error even after passing custom UUID as run_id to airflow dagrun.:

run id already exists

In order to create multiple DagRun within one second, One should set parameter "replace_microseconds" to "false" according to documentation. But how to achieve the same using CLI commands? We are using Amazon Managed Workflows for Apache Airflow (MWAA)

Paresh
  • 3
  • 2

1 Answers1

1

In Airflow CLI, you can specify the execution date and the dag run id, so no need for replace_microseconds, you can try something like:

RUN_TIMESTAMP=$(date +"%Y-%m-%dT%H:%M:%S%z") DAG_NAME="my_dag" airflow dags trigger -e $RUN_TIMESTAMP -r "manual__$RUN_TIMESTAMP" $DAG_NAME
Hussein Awala
  • 4,285
  • 2
  • 9
  • 23
  • Thanks Hussein for this update! I too face this issue in the MWAA CLI after passing both execution date and custom run id, Airflow still throws the same exception "Dag Run already exists" I passed the execution date like this execution_date = datetime.utcnow().strftime("%Y-%m-%dT%H:%m:%S.%f") "dags trigger {0} -c '{1}' -r {2} -e {3}".format(workflow_id, conf, dag_run_id, execution_date) traceback: airflow.exceptions.DagRunAlreadyExists: A Dag Run already exists for dag id NLAC_STEP1_Conduit_Failures at 2022-10-18T06:10:28+00:00 with run id CL4Adauihkvz121928332658Gp6bsTWU – Ankur Vyas Oct 18 '22 at 07:12