Injecting vault secrets into Kubernetes Pod Environment variable

Question

I'm trying to install Sonarqube in Kubernetes environment which needs PostgresSQL. I'm using an external Postgres instance and I have the crednetials kv secret set in Vault. SonarQube helm chart creates an Environment variable in the container which takes the username and password for Postgres.

How can I inject the secret from my Vault to environment variable of sonarqube pod running on Kubernetes?

Creating a Kubernetes secret and using the secret in the helm chart works, but we are managing all secrets on Vault and need Vault secrets to be injected into pods.

Thanks

Answer 1

There are 2 ways to inject vault secrets into the k8s pod as ENV vars.

1) Use the vault Agent Injector

A template should be created that exports a Vault secret as an environment variable.

spec:
  template:
    metadata:
      annotations:
        # Environment variable export template
        vault.hashicorp.com/agent-inject-template-config: |
          {{ with secret "secret/data/web" -}}
            export api_key="{{ .Data.data.payments_api_key }}"
          {{- end }}

And the application container should source those files during startup.

args:
  ['sh', '-c', 'source /vault/secrets/config && <entrypoint script>']

Reference: https://www.vaultproject.io/docs/platform/k8s/injector/examples#environment-variable-example

2) Use banzaicloud bank-vault

Reference: https://banzaicloud.com/blog/inject-secrets-into-pods-vault-revisited/.

Comments:

Both methods are bypassing k8s security because secrets are not stored in etcd. In addition, pods are unaware of vault in both methods. So any one of these can be adopted without a deep comparison.

For vault-k8s and vault-helm users, I recommend the first method.

Answer 2

Option 1 Official Vault

Full answer : https://stackoverflow.com/a/73046067

annotations:
        vault.hashicorp.com/agent-image: <Agent image>
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-inject-secret-secrets: kv/<Path-of-secret>
        vault.hashicorp.com/agent-inject-template-secrets: |2

          {{- with secret "kv/<Path-of-secret>" -}}

          #!/bin/sh
          set -e

          {{- range $key, $value := .Data.data }}
          export {{ $key }}={{ $value }}
          {{- end }}

          exec "$@"
          {{- end }}
        vault.hashicorp.com/auth-path: auth/<K8s cluster for auth>
        vault.hashicorp.com/role: app

Option 2 Third party

If you are facing issue in injecting secret using consul sidecar container and finding it very difficult to setup you can use this : https://github.com/DaspawnW/vault-crd

This is vault-custom resource definition which directly sync vault environment variables to kuberntes secret so now you can directly add secret to POD. with secretref.

vault crd create one pod in which you have to pass vault service name or URL using which application can connect to vault and on changes in vault value it will automatically sync value to kubernetes secret.

https://vault.koudingspawn.de/

Answer 3

You need to use a parent process that will talk to vault and retrieve the value, and then run your real process. https://github.com/hashicorp/envconsul is the marginally official tool for this from the Vault team, but there are many other options if you go looking.

Answer 4

I want to propose a solution which doesn't require hijacking the entrypoint of the container, which is difficult to generalize across deployments, and requires manual intervention each time.

It is quite similar to the accepted answer.

First, you render your secret with an export-statement:

spec:
  template:
    metadata:
      annotations:
        # Environment variable export template
        vault.hashicorp.com/agent-inject-template-config: |
          {{ with secret "secret/data/web" -}}
            export api_key="{{ .Data.data.payments_api_key }}"
          {{- end }}

Then you can override the pods BASH_ENV to make sure this file is sourced at startup:

template:
  spec:
    containers:
      - env:
        - name: BASH_ENV
          value: /vault/secrets/config

This works because the BASH_ENV variable determines which file is sourced at startup of a non-login shell [0]

If you need to export several variables, you need to get a bit more creative with templating, or setting the BASH_ENV variable, but it should be possible.

References:

0: https://www.gnu.org/software/bash/manual/bash.html#index-BASH_005fENV

Answer 5

Here's a method that could provide some insight: https://banzaicloud.com/blog/inject-secrets-into-pods-vault-revisited/#kubernetes-mutating-webhook-for-injecting-secrets

Answer 6

In my case, my deployment had the following annotation:

spec:
  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-configmap: my-configmap
        vault.hashicorp.com/agent-extra-secret: something
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-pre-populate-only: "true"

In my case, I needed to load the variables from /vault/secrets/myfile.env.

I tried using ['sh', '-c', 'source /vault/secrets/myfile.env && <entrypoint script>'], with no luck. What worked for me was using ['sh', '-c', 'export $(cat /vault/secrets/myfile.env | xargs) && <entrypoint script>'].