4

I sometimes get this log for tasks:

[2022-10-03, 00:34:03 UTC] {taskinstance.py:1034} INFO - Dependencies not met for <TaskInstance: dag_name.task_name scheduled__2022-10-02T00:00:00+00:00 [running]>, dependency 'Task Instance Not Running' FAILED: Task is in the running state
[2022-10-03, 00:34:03 UTC] {taskinstance.py:1034} INFO - Dependencies not met for <TaskInstance: dag_name.task_name scheduled__2022-10-02T00:00:00+00:00 [running]>, dependency 'Task Instance State' FAILED: Task is in the 'running' state.

It happens for sensors or normal tasks.

It also happens for the airflow_monitoring dag provided by google.

It is annoying because the task gets stuck until I restart it manually. Is it something I can fix with a configuration change?

Configuration:

Google Cloud Composer 1.19.1
Airflow 2.2.5

Running on Kubernetes cluster with Celery executor

Quentin Sommer
  • 195
  • 2
  • 14
  • can you add more details, airflow version ? what kind of Operators are you using ? local ? docker-compose or any infra you use. 10x – ozs Oct 17 '22 at 10:25
  • Hi, I added the config in the question, as for the operators we see the issue on several different operators, including the very simple `airflow_monitoring` dag provided by google which is just a bash operator with `echo test`. Otherwise we mainly use sensors, Bash, Python, Kubernetes, and various BigQuery operators – Quentin Sommer Oct 17 '22 at 18:03

0 Answers0