0

What configuration is needed to use helm within a k8s job? The error given is x509: certificate signed by unknown authority. What is needed to verify the certificates?

I am trying to use helm CLI tool within a k8s job using the alpine/helm. My cluster is running locally using minikube. For testing, I am trying to simply list all helm charts. Regular kubectl commands work on the cluster within a job. I suspect there is something special needed to configure to the CA certificate and have it be accessible by helm.

apiVersion: batch/v1
kind: Job
metadata:
  name: helm-job
spec:
  template:
    spec:
      containers:
        - name: helm
          image: alpine/helm
          imagePullPolicy: Always
          env:
            - name: HELM_KUBEAPISERVER
              value: "https://kubernetes.default.svc"
            - name: HELM_DEBUG
              value: "true"
          command: [ "helm", "list" ]
      restartPolicy: Never
      automountServiceAccountToken: false

If automountServiceAccountToken is not set to false the Job crashes with the following error on start up: MountVolume.SetUp failed for volume "kube-api-access-rqc6l" : object "default"/"kube-root-ca.crt" not registered. When it is set to false the job has the following debug logs:

Error: Kubernetes cluster unreachable: Get "https://kubernetes.default.svc/version": x509: certificate signed by unknown authority
helm.go:84: [debug] Get "https://kubernetes.default.svc/version": x509: certificate signed by unknown authority
Kubernetes cluster unreachable
helm.sh/helm/v3/pkg/kube.(*Client).IsReachable
helm.sh/helm/v3/pkg/kube/client.go:121
helm.sh/helm/v3/pkg/action.(*List).Run
helm.sh/helm/v3/pkg/action/list.go:148
main.newListCmd.func1
helm.sh/helm/v3/cmd/helm/list.go:80
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.3.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.3.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.3.0/command.go:902
main.main
helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
runtime/proc.go:255
runtime.goexit
runtime/asm_amd64.s:1581

Unfortunately, helm: x509: certificate signed by unknown authority is not a solution. The answers given using helm init is for helm v2 not v3. When setting insecure-skip-tls-verify: true the following error is thrown: Unable to restart cluster, will reset it: getting k8s client: specifying a root certificates file with the insecure flag is not allowed.

Edit:

The default service account is available on all my nodes. I did also try create separate service accounts.

Blunderchips
  • 534
  • 4
  • 22
  • You are not able to mount the default service account token cause it is not available in all your nodes. You can create a separate service account.. https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ – DanielM Mar 27 '22 at 23:08
  • @DanielM That was my first guess, afraid it is not a solution. The default service account is available on all my nodes. I did also try create separate service accounts. As stated in the question, I suspect there is something special needed to configure to the CA certificate and have it be accessible by helm. – Blunderchips Apr 03 '22 at 09:16

0 Answers0