0

I am creating a Airflow DAG in GCC that needs to read a Secret and no other DAG in that instance should access it besides that particular DAG.

Is that possible? I only see that we could grant permissions for the GCC SA, in this case every DAG in the GCC instance could read the secret.

Is there some alternative for reading secrets with that constraint? Am I reading correctly the situation?

luisvenezian
  • 441
  • 5
  • 18
  • 1
    I'm not a Composer expert, but I think you can set a specific "connection" to a DAG. That connection define a credential to Google Cloud, and so, grant only that connection the permission to access the secret. https://cloud.google.com/composer/docs/how-to/managing/connections – guillaume blaquiere Aug 18 '22 at 18:51

1 Answers1

1

To connect your airflow server (GCC or other services) to Google Secret Manager, you have multiple options:

  • use Google Secret Manager as a secret backend for airflow: in this case, each Variable.getor <some hook>.get_connection will check if if the variable or the connection exists in Google Secret Manager, regardless of the DAG who calls these methods.
  • use SecretsManagerHook: you need to create an airflow connection to store the service account credentials, then you can use the hook wherever you want, so you will have the ability to decide where you want to use the secret manager and where you don't want to use it, and to read the secrets you can use the method get_secret
Hussein Awala
  • 4,285
  • 2
  • 9
  • 23