0

I've tried to setup a cron job to patch a horizontal pod autoscaler, but the job returns horizontalpodautoscalers.autoscaling "my-web-service" is forbidden: User "system:serviceaccount:staging:sa-cron-runner" cannot get resource "horizontalpodautoscalers" in API group "autoscaling" in the namespace "staging"

I've tried setting up all the role permissions as below:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: staging
  name: cron-runner
rules:
- apiGroups: ["autoscaling"]
  resources: ["horizontalpodautoscalers"]
  verbs: ["patch"]

---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: cron-runner
  namespace: staging
subjects:
- kind: ServiceAccount
  name: sa-cron-runner
  namespace: staging
roleRef:
  kind: Role
  name: cron-runner
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-cron-runner
  namespace: staging
---

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: scale-up-job
  namespace: staging
spec:
  schedule: "16 20 * * 1-6"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: sa-cron-runner
          containers:
          - name: scale-up-job
            image: bitnami/kubectl:latest
            command:
            - /bin/sh
            - -c
            - kubectl patch hpa my-web-service --patch '{\"spec\":{\"minReplicas\":6}}'
          restartPolicy: OnFailure

kubectl auth can-i patch horizontalpodautoscalers -n staging --as sa-cron-runner also returns no, so the permissions can't be setup correctly.

How can I debug this? I can't see how this is setup incorrectly

MDalt
  • 1,681
  • 2
  • 24
  • 46
  • Does this answer your question? [Kubernetes check serviceaccount permissions](https://stackoverflow.com/questions/54889458/kubernetes-check-serviceaccount-permissions) – Manuel Feb 09 '21 at 20:35
  • Hmm how would that apply to the above?...I've done all that as far as I can see – MDalt Feb 10 '21 at 07:46
  • It's the way how you're asking for. --as=system:serviceaccount:staging:sa-cron-runner. I guess this is what you're looking for. The alternative to ask for on verb is to get the whole list like kubectl auth can-i --list --as=system:serviceaccount:staging:sa-cron-runner. But i see now in your output that it already uses the correct serviceaccount. – Manuel Feb 10 '21 at 08:19

1 Answers1

3

I think that the problem is that the cronjob needs to first get the resource and then Patch the same. So, you need to explicitly specify the permission to get in the Role spec.

The error also mentions authentication problem with getting the resource:

horizontalpodautoscalers.autoscaling "my-web-service" is forbidden: User "system:serviceaccount:staging:sa-cron-runner" cannot get resource "horizontalpodautoscalers" in API group "autoscaling" in the namespace "staging"

Try modifying the verbs part of the Role as:

verbs: ["patch", "get"]

You could also include list and watch depending on the requirements in the scripts that is run inside the cronjob.

Krishna Chaurasia
  • 8,924
  • 6
  • 22
  • 35
  • Could be _a_ problem, but don't think that's the auth problem. `kubectl auth can-i patch horizontalpodautoscalers -n staging --as sa-cron-runner ` still returns no. – MDalt Feb 10 '21 at 07:43
  • try using like: `kubectl auth can-i patch horizontalpodautoscalers -n staging --as system:serviceaccount:staging:sa-cron-runner` – Krishna Chaurasia Feb 10 '21 at 07:59
  • After adding this, and using the command Krishan suggested, it worked! – MDalt Feb 10 '21 at 14:55