14

I see that one can trigger_dag with parameters/config key-value pairs using the airflow command line:

For Apache Airflow, How can I pass the parameters when manually trigger DAG via CLI?

In my case, I would like to centralize all operations for airflow via the airflow UI (preferably no CLI access should be granted), which has a button for triggering the dags:

enter image description here

How can I pass the external parameters/configs via the UI?

chaooder
  • 1,358
  • 1
  • 17
  • 37

2 Answers2

9

You're in luck, assuming you're on a recent version of Airflow or can upgrade. Support for triggering a DAG run with a config blob was added in Airflow 1.10.8 under AIRFLOW-5843 in #5843. It's not through the same Trigger DAG icon you've pointed to, but it's through creating a DAG Run from Browse->DAG Runs->Create.

Screenshot of the new form that supports conf copied below from the pull request that added it.

Add DAG Run

The form will also validate the JSON of the conf which may actually be a step beyond the CLI.

Daniel Huang
  • 6,238
  • 34
  • 33
  • 1
    Thanks! I'm actually using cloud composer in GCP, and currently the latest version is 1.10.6. I guess we'll have to wait till Google releases newer versions of airflow in cloud composer. – chaooder Mar 07 '20 at 04:13
  • 4
    I'm using 1.10.9 version and the `Conf` option doesn't exist. – 4ndt3s Mar 26 '20 at 01:44
  • 3
    I'm using 1.10.10 version and the Conf option doesn't exist either – Saffik May 23 '20 at 10:36
  • 3
    You people want 1.10.12 or higher ;) – Mark K Cowan Aug 28 '20 at 09:19
  • 1
    I saw the same problem of missing the conf option, and now understand why. As of 1.10.12, the conf option appears only when RBAC UI is enabled. Once I turned it on, I was able to find the field on GUI even the version is 1.10.8. http://airflow.apache.org/docs/stable/configurations-ref.html#rbac – ohbarye Oct 15 '20 at 04:05
8

With latest versions [we use 2.2.2] of Airflow we only pass config json

When we click on the Trigger/Play button available against the DAG, we get a drop down with 2 options

  1. Trigger DAG
  2. Trigger DAG w/ config

enter image description here Clicking on the second option you can pass the conf in json format and that can be used in the script and its usage can be viewed while you create the configuration JSON as below.

To access configuration in your DAG use {{ dag_run.conf }}. As core.dag_run_conf_overrides_params is set to False, so passing any configuration here won't override task params.

enter image description here

Ramji
  • 101
  • 1
  • 4