Read Airflow UI config file in python code and use the values as parameter

Question

I am new to Airflow. I have a DAG.py wherein I am using a BashOperator to run a python script.

I would like to pass some argument for date in this python script using a config file. I see that one can trigger a DAG with a config on the UI:

I am not sure how I can read this in my DAG.py and pass that to the python script for using as a parameter.

My DAG.py looks like this:

from airflow import DAG
from airflow.operators.bash import BashOperator


dag = DAG(
      'Sample_DAG', 
      description='DAG for Sample run',
          schedule_interval='@monthly',
          start_date=datetime(2022, 11, 15),
          catchup=False,
          )

bash_op = BashOperator(
    task_id="run_hello_file",
    bash_command= "python3.6 /path/to/hello_world.py",
    dag=dag
)
bash_op

Apologies if its a very trivial question but I could not find anything which answers my question. Most of the answers just tell how to use the config but not how to read it using python and pass it as a parameter

Answer 1

If you will trigger DAG with config as:

{"my_key":"hello"}

and your code is:

bash_command= "echo {{ dag_run.conf['my_key'] }}"

it will print hello

In your case you can use it to pass parameters to your script:

bash_command= "python3.6 /path/to/hello_world.py {{ dag_run.conf['my_key'] }}",

Example:

Will be rendered as:

You can also configure Params if you want default parameters for scheduled runs but to allow override them in manual runs.