Kustomize: Can I managed using single overlay per environment (dev/test/qa) where environment variables for each microservice are different

Question

I have different sets of environment variables per deployment/microservice and vaule for each environment (dev/test/qa) are different.

Do I need overlay file for each deployment/microservice against each environment (dev/test/qa) or I can managed with single overlay per environment?

Deployment - app-1.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-1
spec:
  template:
    spec:
      containers:
        - name: example-1
        image: example:1.0
        env:
          -Name: "A1"
          -Value: "B1"
        env:
          -Name: "D1"
          -Value: "E1"

Deployment - app-2.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-2
spec:
  template:
    spec:
      containers:
        - name: example-2
        image: example:2.0
        env:
          -Name: "X1"
          -Value: "Y1"
        env:
          -Name: "P1"
          -Value: "Q1"

Answer 1

You can keep everything inside the single YAML file and divide the YAML as per need.

you can use the --- to merge the YAML configuration files in one file like the below in the example.

In a single YAML file, you can add everything Secret, deployment, service and etc as per requirement.

However if for each environment you have a different cluster to manage then applying the single YAML file might create the issues, in that case, it's better to keep the files separate.

If you are planning to set up the CI/CD and automation for deployment i would suggest keeping the single deployment file with a variable approach.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: DEPLOYMENT_NAME
spec:
  template:
    spec:
      containers:
        - name: example-2
        image: IMAGE_NAME
        env:
          -Name: "A1"
          -Value: "B1"

using the ubuntu sed command you have to run time to replace the values of IMAGE_NAME, DEPLOYMENT_NAME and A1 etc based on the environment and as soon as your file get ready you can apply from the CI/CD server

Single merged file :

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-1
spec:
  template:
    spec:
      containers:
        - name: example-1
        image: example:1.0
        env:
          -Name: "A1"
          -Value: "B1"
        env:
          -Name: "D1"
          -Value: "E1"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-2
spec:
  template:
    spec:
      containers:
        - name: example-2
        image: example:2.0
        env:
          -Name: "X1"
          -Value: "Y1"
        env:
          -Name: "P1"
          -Value: "Q1"

EDIT

If managing the environment is the only case you can also use the secret or configmap to manage the environment variables.

secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: dev-env-sample
  annotations:
    kubernetes.io/service-account.name: "sa-name"
type: kubernetes.io/service-account-token
data:
  extra: YmFyCg==
  A1 : B1
  D1 : E1

this is a single secret file storing all the Dev environment variables.

inject all variables to the Dev deployment, add the below config to the deployment file so all your variables inside that config map or secret will get injected into the deployment.

envFrom:
        - SecretRef:
            name: dev-env-sample

https://kubernetes.io/docs/concepts/configuration/secret/

configmap.yaml

kind: ConfigMap 
apiVersion: v1 
metadata:
  name: dev-configmap 
data:
  extra: YmFyCg==
  A1 : B1
  D1 : E1

and you can inject the configmap to deployment using the

envFrom:
            - configMapRef:
                name: dev-configmap 

The difference between secret and configmap is that secret save the data in base64 encoded format while configmap save in plain text.

You can also merge the multiple secrets or config map in single YAML file

apiVersion: v1
    kind: Secret
    metadata:
      name: dev-env-sample
      annotations:
        kubernetes.io/service-account.name: "sa-name"
    type: kubernetes.io/service-account-token
    data:
      extra: YmFyCg==
      A1 : B1
      D1 : E1
---
apiVersion: v1
    kind: Secret
    metadata:
      name: stag-env-sample
      annotations:
        kubernetes.io/service-account.name: "sa-name"
    type: kubernetes.io/service-account-token
    data:
      extra: YmFyCg==
      A1 : B1
      D1 : E1
---
apiVersion: v1
    kind: Secret
    metadata:
      name: prod-env-sample
      annotations:
        kubernetes.io/service-account.name: "sa-name"
    type: kubernetes.io/service-account-token
    data:
      extra: YmFyCg==
      A1 : B1
      D1 : E1

inject the secret into deployment as per need per environment.

dev-env-sample can be added in to the deployment file for dev environment.

Answer 2

You can use variable in the env value field as below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-1
spec:
  template:
    spec:
      containers:
      - name: example-1
        image: example:1.0
        env:
        - Name: "A1"
          Value: ${A1_VALUE}
        - Name: "D1"
          Value: ${D1_VALUE}

Then, on dev env you can do the following.

export A1_VALUE=<your dev env value for A1>
export D1_VALUE=<your dev env value for D1>

envsubst < example-1.yaml | kubectl apply -f -

Then, on qa env you can do the following.

export A1_VALUE=<your qa env value for A1>
export D1_VALUE=<your qa env value for D1>

envsubst < example-1.yaml | kubectl apply -f -

You can also put those env variables in a file. For example, you can have the following two env files.

dev.env file

A1_VALUE=a1_dev
D1_VALUE=b1_dev

qa.env file

A1_VALUE=a1_qa
D1_VALUE=b1_qa

Then, on dev environment, just run:

❯ source dev.env
❯ envsubst < example-1.yaml| kubectl apply -f -

On qa environment, just run:

❯ source qa.env
❯ envsubst < example-1.yaml| kubectl apply -f -

Note that you have to install envsubst in your machine.