0

I have a DAG that fetches stock data after market hours 16:30 EST. I need to fetch the same date as the date parameter in my task to fetch 1 year old stock data. I initially used {{ds}} and found out {{ds}} only passed the interval before the run date by design. (ie, a DAG I schedule to run on 2022-10-11 with daily @ 16:30 EST interval will have a {{ds}} == '2022-10-10'. I found {{ data_interval_end }} may do the trick but get the following error since {{data_interval_end}} is in datetime format. Is there way I could format to "YYYY-MM-DD"" within the Python Operator?

ValueError: unconverted data remains: T20:45:00+00:00

     execution_date = "{{ data_interval_end }}"
    
    
    
    with DAG("DAG_SAMPLE",
        start_date=pendulum.datetime(2021, 1 ,1, tz="US/Pacific"),
        schedule_interval='45 13 * * 1-5', #At 01:45 PM, Monday through Friday
        catchup=False) as dag:
    
  
        scanning_swing_pattern_task = PythonOperator(
            task_id='scanning_swing_pattern_task',
            python_callable=scanning_swing_pattern.main,
            op_kwargs={'run_date': execution_date}
            )
NinjaWarrior
  • 25
  • 1
  • 2
  • 9

0 Answers0