Airflow DAG Not following schedule

Question

I have a DAG that is scheduled once a month. my problem is that the scheduler is not kicking off the job:

args = {
    'owner': 'Airflow',
    'start_date': dates.days_ago(1),
    'email': ['sinistersparrow1701@gmail.com', 'rich@offrs.com'],
    'email_on_failure': True,
    'email_on_success': True,
}

dag = models.DAG(
    dag_id='Offrs_TAX_ASSESSOR_MASTER_Production',
    default_args=args,
    schedule_interval= '0 0 1 * *',
    catchup=False
)

I tried 0 0 1 * * and I tried @Monthly. It is not firing. The DAG works perfectly when executed manually. Do I have something set wrong?

Thanks!

score 1 · Answer 1 · answered May 03 '21 at 18:33

Do not use dynamic values in start_date.

args = {
    'owner': 'Airflow',
    'start_date': datetime(2021, 4, 1),
    'email': ['sinistersparrow1701@gmail.com', 'rich@offrs.com'],
    'email_on_failure': True,
    'email_on_success': True,
}

In simple words Airflow calculates start_date+schedule_interval and execute the task at the end of the interval. When you use dynamic start_date you might get a case where the interval never end as the base point is always moving. This is explained in depth at the docs.

Airflow DAG Not following schedule

1 Answers1