0

I have perfectly configured Airflow v2.4.1 application working over Docker, which when upgraded to v2.4.2 or higher fails to load DAGs. I tried every possible solutions but no use.

When playing around with airflow.cfg file, considering the error shown, I found out that changing the log level from "INFO" to "ERROR" fixed the issue and all DAGs can be seen on UI also the rest is working fine.

I need to understand the root cause of the issue, so that I can fix it or raise a PR if required.

Details: Airflow version 2.5.0 40+ DAGs Docker image: debian11-0.0.4 Python version: python:3.9

Error:

File "/usr/local/lib/python3.9/site-packages/pendulum/tz/timezone.py", line 320, in _normalize
  dt = dt.__class__(
RecursionError: maximum recursion depth exceeded while calling Python object

I have tried updating the Recursion limit using sys.setrecursionlimit() to max 100000.

Hille
  • 2,123
  • 22
  • 39
PrashanjeetH
  • 3
  • 1
  • Long shot but are you using timezone settings in your dags? Does the error persist when only using dags with start_dates in UTC? – TJaniF Jan 19 '23 at 10:00
  • Yes, the timezone is an important factor for my DAGs. but the error is not consistent: sometimes the error is as: File: "/usr/local/lib/python3.9/posixpath.py", line 143, in basename sep = _get_sep(p) File: "/usr/local/lib/python3.9/posixpath.py", line 42, in basename if isinstance(path, bytes): RecursionError: maximum recursion depth exceeded while calling a Python object – PrashanjeetH Jan 19 '23 at 11:10
  • It will probably be really hard to get to a root cause unless it can be explicitly reproducible. It might be worthwhile to [open a discussion](https://github.com/apache/airflow/discussions/) on the Airflow GitHub. – RNHTTR Jan 19 '23 at 15:13

0 Answers0