Conftest Exception Rule Fails with Kustomization & Helm

Question

I'm having in one of my projects several k8s resources which is built, composed and packaged using Helm & Kustomize. I wrote few OPA tests using Conftest where one of the check is to avoid running containers as root. So here is my deployment.yaml in my base folder:

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: {{ .Values.app.namespace }}
  labels:
    name: {{ .Values.app.name }}
    #app: {{ .Values.app.name }}
    component: {{ .Values.plantSimulatorService.component }}
    part-of: {{ .Values.app.name }}
    managed-by: helm
    instance: {{ .Values.app.name }}
    version: {{ .Values.app.version }}
spec:
  selector:
    matchLabels:
      app: {{ .Values.app.name }}
  replicas: 1
  template:
    metadata:
      labels:
        app: {{ .Values.app.name }}
    spec:
      securityContext:
        runAsUser: 1000
        runAsGroup: 3000
        fsGroup: 2000
        runAsNonRoot: true
      containers:
        - name: {{ .Values.app.name }}
          image: {{ .Values.plantSimulatorService.image.repository }}:{{ .Values.plantSimulatorService.image.tag }}
          ports:
            - containerPort: {{ .Values.plantSimulatorService.ports.containerPort }} # Get this value from ConfigMap

I then have a patch file (flux-patch-prod.yaml) in my overlays folder that looks like this:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    fluxPatchFile: prodPatchFile
  annotations:
    flux.weave.works/locked: "true"
    flux.weave.works/locked_msg: Lock deployment in production
    flux.weave.works/locked_user: Joesan <github.com/joesan>
  name: plant-simulator-prod
  namespace: {{ .Values.app.namespace }}

I have now written the Conftest in my base.rego file that looks like this:

# Check container is not run as root
deny_run_as_root[msg] {
  kubernetes.is_deployment
  not input.spec.template.spec.securityContext.runAsNonRoot

  msg = sprintf("Containers must not run as root in Deployment %s", [name])
}

exception[rules] {
  kubernetes.is_deployment
  input.metadata.name == "plant-simulator-prod"
  rules := ["run_as_root"]
}

But when I ran them (I have the helm-conftest plugin installed), I get the following error:

FAIL - Containers must not run as root in Deployment plant-simulator-prod
90 tests, 80 passed, 3 warnings, 7 failures
Error: plugin "conftest" exited with error

I have no idea how to get this working. I do not want to end up copying the contents from deployment.yaml into the flux-patch-prod.yaml back again as it would defeat the whole purpose of using Kustomization in the first place. Any idea how to get this fixed? I have been down with this issue since yesterday!

score 0 · Accepted Answer · answered Dec 11 '20 at 14:02

I managed to get this fixed, but the error messages that were throwing up were not that so helpful. May be it gets better with next release versions of Conftest.

So here is what I had to do:

I went to the https://play.openpolicyagent.org/ playground to test out my files. As I copied my rego rules over there, I noticed the following error message:

policy.rego:25: rego_unsafe_var_error: var msg is unsafe

I got suspicious and as I paid closer attention to the line to see what the actual problem is, I had to change from:

msg = sprintf("Containers must not run as root in Deployment %s", [name])

To:

msg = sprintf("Containers must not run as root in Deployment %s", [input.name])

And it worked as expected! Silly mistake, but the error messages from Conftest were not that useful to figure them out in the first glance!

Conftest Exception Rule Fails with Kustomization & Helm

1 Answers1