running airflow 2.2.5 on python3.6, I get this error after running a dag from console:
airflow dags test airflow_report1_email 2022-08-30
...
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 826, in _execute
session=session,
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/session.py", line 67, in wrapper
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 739, in _execute_dagruns
session=session,
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/session.py", line 67, in wrapper
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 634, in _process_backfill_task_instances
self._update_counters(ti_status=ti_status)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 216, in _update_counters
ti_status.running.pop(reduced_key)
KeyError: TaskInstanceKey(dag_id='airflow_report1_email', task_id='list_all_files', run_id='backfill__2022-08-30T00:00:00+00:00', try_number=12)
my dag is very simple:
from datetime import datetime, timedelta
from textwrap import dedent
from airflow import DAG
from airflow.operators.bash import BashOperator
with DAG(
'airflow_report1_email',
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
schedule_interval='0 12 * * *',
default_args={
'depends_on_past': True,
'email': ['REDACTED'],
'email_on_failure': True,
'email_on_retry': True,
'retries': 1,
'retry_delay': timedelta(minutes=5),
},
description='DAG to report the health of data',
start_date=datetime(2022, 8, 30),
catchup=False,
tags=['example'],
) as dag:
t1 = BashOperator(
task_id='list_all_files',
bash_command='/list_all_files.sh ',
retries=1,
)
t1.doc_md = dedent(
"""\
#### list_all_files
This task downloads the (1) list of all files, (2) for each file for the past week checks if it's new,
(3) downloads the new file.
Estimated time to run: ~1h
"""
)
t2 = BashOperator(
task_id='report_email',
bash_command='/report1_email.sh ',
retries=1,
)
t2.doc_md = dedent(
"""\
#### report1_email
This task computes the health of the newly downloaded data (last day of data).
Estimated time to run: 5min
"""
)
dag.doc_md = """
This is a DAG to report the health of data (project)
"""
t1 >> t2
Good to mention that I have the "airflow" user, and we had to open up permissions and create common groups to allow the airflow user to operate. Also, I found this very similar. Maybe I need too upgrade to something >= 2.3.3 as this seems to be addressed.
