2

I'm deploying Airflow 2 on GKE Autopilot using helm chart and have provisioned a Cloud SQL instance (MySQL) to be used as DB by airflow.

I have created (using kubectl) a secret in K8s with this connection string as value and wanted to give that as an env var to all airflow pods. So tried to provide that in

env: []

section of this chart (line no 239), but it can not use valueFrom attribute there. It need value. So I want to know what are the ways by which I can refer to a secret in this helm chart and provide that as env var value to all the containers this chart deploys

Rohit Shisode
  • 142
  • 8

2 Answers2

3

Answering my own for others to find correct solution -

  1. Create the secret with connection key and value as database URI
  2. Disable postgres deployment in helm values.yaml
  3. Change data.metadataSecretName to the secret create in #1. Airflow will pick up and inject that as connection URI

Answer by Harsh Manvar is still valid and correct one, but that is more suited for injecting arbitrary secrets as env vars. For changing database and providing custom URI, approach I took is recommended - https://airflow.apache.org/docs/helm-chart/stable/production-guide.html#database

Rohit Shisode
  • 142
  • 8
1

You can checkout the line no. 244 which is injecting the secret to all PODs

it will also i think do same think as we can inject the secret as env variable so.

# Secrets for all airflow containers
secret: []
# - envName: ""
#   secretName: ""
#   secretKey: ""

values.yaml : https://github.com/apache/airflow/blob/main/chart/values.yaml#L243

Documentation details : https://github.com/apache/airflow/blob/main/docs/helm-chart/adding-connections-and-variables.rst#connections-and-sensitive-environment-variables

Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102
  • Yes, I too stumbled onto this and tried it out, but this did not workout. I think this is okay for injecting arbitrary secrets as env vars in airflow pods but for database connection we have to set data.metadataSecretName to the secret we create and the secret must have a key - **connection** in it. Doc - https://airflow.apache.org/docs/helm-chart/stable/production-guide.html#database – Rohit Shisode Jun 12 '22 at 13:09