Kustomize: how to reference a value from a ConfigMap in another resource/overlay?

Question

I have a couple of overlays (dev, stg, prod) pulling data from multiple bases where each base contains a single service so that each overlay can pick and choose what services it needs. I generate the manifests from the dev/stg/prod directories.

A simplified version of my Kubernetes/Kustomize directory structure looks like this:

├── base
│   ├── ServiceOne
│   │   ├── kustomization.yaml
│   │   └── service_one_config.yaml
│   ├── ServiceTwo
│   │   ├── kustomization.yaml
│   │   └── service_two_config.yaml
│   └── ConfigMap
│       ├── kustomization.yaml
│       └── config_map_constants.yaml
└── overlays
    ├── dev
    │   ├── kustomization.yaml
    │   └── dev_patch.yaml
    ├── stg
    │   ├── kustomization.yaml
    │   └── stg_patch.yaml
    └── prod
        ├── kustomization.yaml
        └── prod_patch.yaml

Under base/ConfigMap, config_map_constants.yaml file contains key/value pairs that are non-secrets:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: myApp
  name: global-config-map
  namespace: myNamespace
data:
  aws_region: "us-west"
  env_id: "1234"

If an overlay just needs a default value, it should reference the key/value pair as is, and if it needs a custom value, I would use a patch to override the value.

kustomization.yaml from base/ConfigMap looks like this and refers to ConfigMap as a resource:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - config_map_constants.yaml

QUESTION: how do I reference "aws_region" in my overlays' yaml files so that I can retrieve the value?

For example, I want to be able to do something like this in base/ServiceOne/service_one_config.yaml:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: myApp
    aws_region: ../ConfigMap/${aws_region} #pseudo syntax
  name: service_one
spec:
  env_id: ../ConfigMap/${env_id} #pseudo syntax

I am able to build the ConfigMap and append it to my services but I am struggling to find how to reference its contents within other resources.

EDIT: Kustomize version: v4.5.2

Which version of Kubernetes did you use and how did you set up the cluster? — kkopczak, Mar 07 '22 at 14:56
The version is v4.5.2 as in the post above. We generate the deployment descriptors, save them in a separate repo and ArgoCD pulls them to set up the deployments. So the main goal here is to simply generate the correct yaml files — unboundedcauchy, Mar 07 '22 at 17:03

score 3 · Accepted Answer · answered Mar 18 '22 at 06:10

You can try using https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/replacements/

For your scenario, if you want to reference the aws-region into your Service labels. You need to create a replacement file.

replacements/region.yaml

source:
  kind: ConfigMap
  fieldPath: data.aws-region
targets:
  - select:
      kind: Service
      name: service_one
    fieldPaths:
      - metadata.labels.aws_region

And add it to your kustomization.yaml

replacements:
  - path: replacements/region.yaml

Kustomize output should be similar to this

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: myApp
    aws_region: us-west-1
  name: service_one

Kustomize: how to reference a value from a ConfigMap in another resource/overlay?

1 Answers1