1

I have several Dags setup to run the KubernetesPodOperator in Cloud Composer / Airflow, they run under the Composer Environment Service Account, I want to know if there is away of running a Dag with a different Service Account to the one set in the Composer environment ?

I have looked thought the documentation but cannot see how this might be done

When you create an environment, you specify a service account. Your environment's cluster uses this service account to run pods with different environment components, such as Airflow workers and schedulers. https://cloud.google.com/composer/docs/how-to/access-control

https://cloud.google.com/composer/docs/composer-2/authentication

I think you might be able to do this with impersonate service account but there are no examples of how to do this https://github.com/apache/airflow/issues/12061

Hutch
  • 411
  • 10
  • 32

1 Answers1

3

If you use GCP operators to interface with Google services, you can do it through different connections.

You can define any number of connections you want that should be 'GCP' connections, and use those connections in your DAGS/tasks. This way you can even have different service account for different tasks in the same DAG.

See https://airflow.apache.org/docs/apache-airflow-providers-google/stable/connections/gcp.html

Jarek Potiuk
  • 19,317
  • 2
  • 60
  • 61