Kustomize structure for different environments and cloud providers

Question

I have a scenario and was wondering the best way to structure it with Kustomize.

Say I have multiple environments: dev, qa, prod

and say I have multiple DCs: OnPrem, AWS, GCP

Let's say each DC above has a dev, qa, prod environment.

I have data that is per environment but also per DC. For example, apply this string to dev overlays but apply these, if AWS.

Is there a way to easily doing this without duplication. An example may be, say if it's AWS, I want to run an additional container in my pod, and if it's prod I want extra replicas. If it's GCP, I want a different image but if it's prod, I still want extra replicas.

The below example, will have a lot of duplication. I've read you can do multiple bases. Maybe it makes sense to have a AWS, GCP, OnPrep Base and then have a dev, qa, prod overlay and have mutiple Kustomize files for each?

ie

├── base
│   ├── guestbook-ui-deployment.yaml
│   ├── guestbook-ui-svc.yaml
│   └── kustomization.yaml
└── overlay
    ├── dev
    │   ├── aws
    │   │   ├── guestbook-ui-deployment.yaml
    │   │   └── kustomization.yaml
    │   └── gcp
    │       ├── guestbook-ui-deployment.yaml
    │       └── kustomization.yaml
    └── qa
        ├── aws
        │   ├── guestbook-ui-deployment.yaml
        │   └── kustomization.yaml
        └── gcp
            ├── guestbook-ui-deployment.yaml
            └── kustomization.yaml

score 1 · Answer 1 · answered Jun 25 '22 at 17:31

I recommend having an overlay for each combination you want to build. e.g:

└── overlays
    ├── aws-dev
    ├── aws-qa
    └── gcp-dev

Then you can structure in different ways, such as using components:

└── components
    ├── environments
    │   ├── dev
    │   └── qa
    └── providers
        ├── aws
        └── gcp

This makes sense because you usually don't create all combinations of possible environments, but only some that make sense to you.

More documentation: https://github.com/kubernetes-sigs/kustomize/blob/master/examples/components.md

Kustomize structure for different environments and cloud providers

1 Answers1