0

I am triggering an airflow dag with an api but it is throwing me this response

{
  "detail": "DAGRun with DAG ID: 'working_1' and DAGRun logical date: '2022-11-03 06:35:14+00:00' already exists",
  "status": 409,
  "title": "Conflict",
  "type": "https://airflow.apache.org/docs/apache-airflow/2.3.3/stable-rest-api-ref.html#section/Errors/AlreadyExists"
}

and this is becasue i am triggering multiple dag runs in 1 seconds with same DAG ID. Below is my code

data = {
    "conf": {},
    "replace_microseconds":"false"
    "dag_run_id": str(uuid.uuid4()),
    "logical_date": str(formatted_date),
}
headers={
    'Content-type':'application/json',
    'Accept':'application/json'
}
json_payload = json.dumps(data)
r = requests.post("http://localhost:8080/api/v1/dags/working_1/dagRuns", auth=HTTPBasicAuth("airflow", "airflow"), data=json_payload, headers=headers)
print(r.status_code)
print(r.text)

i tried looking at at the airflow documentation where they have mentioned that using "replace_microseconds" and setting it to false will get the job done. But, it doesn't solved my problem. Also, becasue of some constraints i cannot put a time.sleep(1) between every dag run. Also there is a fix datetime format of logical_date so i am not able to add the microseconds into it.

If anyone can help me what sort of chage will get the job done will be great

NeedToCodeAgain
  • 129
  • 2
  • 9

0 Answers0