I have been configuring Airflow Helm using the official documentation found at the following link: https://airflow.apache.org/docs/helm-chart/stable/production-guide.html.
In my Airflow Helm configuration (values.yaml), I encountered a conflict between using metadataSecretName and integrating pgbouncer, and I'm not exactly sure why this is happening.
In the values.yaml file, I have configured the secret containing the PostgreSQL database connection information for Airflow using
# Airflow database (extern)
data:
metadataSecretName: airflow-postgres-secret
# Disable the deployment of the PostgreSQL container included in the chart
postgresql:
enabled: false
Further down in the same values.yaml file, I have also configured PgBouncer to be enabled and specified the secret containing PgBouncer connection information using
# PgBouncer configuration
pgbouncer:
enabled: true
configSecretName: airflow-pgbouncer-secret
The problem arises when PgBouncer is enabled with the above configuration. It seems that PgBouncer is unable to properly retrieve the connection information from the secret.
To overcome the conflict, I tried using metadataConnection directly to provide the connection details, like this:
metadataConnection:
user: airflow
pass: my-password
protocol: postgresql
host: pg-host
port: 5432
db: airflow
sslmode: disable
This approach resolved the connection issues, but it has the downside of exposing the password directly in the values.yaml file, which is not secure and can lead to potential vulnerabilities.
I am seeking guidance on how to overcome the conflict between metadataSecretName and PgBouncer so that I can securely store the connection information without exposing the password in the values.yaml file.
