0

We're testing Shiny-proxy Kubernetes containers and each application spins it's own container, it's working fine until this part. We have made some changes to create a PVC/PV to persist user specific data for each container, noticed that serviceaccount is unable to create the PVC though I have following roles configured for the account. In general, are there any other steps for making sure that SA is able to access/create PVC?

The PV/PVC are accessible when testing from a normal container, but seem to be an issue with the serviceaccount roles/permissions that's used to create new containers.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: sp-ns
  name: sp-sa
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "persistentvolumeclaims"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

I have verfied that the serviceaccount roles are set right as below commands returns 'yes'.

kubectl auth can-i create pvc --as=system:serviceaccount:sp-ns:sp-sa -n sp-ns

Error during container creation from the application:

at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: http://localhost:8001/api/v1/namespaces/sp-ns/pods. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "sp-pod-92e1efc0-0859-4a87-8b9b-04d6adaa11f5" is forbidden: user "system:serviceaccount:sp-ns:sp-sa" is not an admin and does not have permissions to use host bind mounts for resource .
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:503)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:440)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:406)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:365)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:234)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:735)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:325)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:321)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.lambda$createNew$0(BaseOperation.java:336)
    at io.fabric8.kubernetes.api.model.DoneablePod.done(DoneablePod.java:26)
    at eu.openanalytics.containerproxy.backend.kubernetes.KubernetesBackend.startContainer(KubernetesBackend.java:223)
    at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.doStartProxy(AbstractContainerBackend.java:129)
    at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.startProxy(AbstractContainerBackend.java:110)
    ... 95 more
Arghya Sadhu
  • 41,002
  • 9
  • 78
  • 107
cnu
  • 461
  • 8
  • 19

1 Answers1

1

  1. Container is not running as privileged. Use privileged: true in pod spec.

  2. Service account don't have cluster-admin role. Use below to provide permission.

    kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=sp-ns:sp-sa

Arghya Sadhu
  • 41,002
  • 9
  • 78
  • 107
  • The container spins up fine if PVC related lines are removed from the code/YAML configuration. Doe we need cluster admin role for accessing a PVC? – cnu Jul 21 '20 at 17:05
  • Yes, you need admin role for accessing a PVC. – Ahmad P Jul 22 '20 at 13:14
  • I couldn't find any documentation on why we need cluster-admin role for accessing a PVC as it's a namespace scope resource, not a cluster scoped resource. I will create a cluster role for the same resource and see if that makes a difference. – cnu Jul 24 '20 at 12:49
  • Cluster-admin role gives full control over every resource in the role binding's namespace, including the namespace itself. Please check [this document](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) for more details. – Ahmad P Jul 27 '20 at 13:45
  • Adding cluster-admin rolebinding solved the issue for creating the pvc. – cnu Jul 29 '20 at 19:28
  • If I recall it right, PVC is a namespace scoped resource, is there a reason why cluster-admin privileges are required for serviceaccount since that's not the case for a human user to access/create a PVC? – cnu Aug 05 '20 at 00:22
  • I am also curious why my namespace scoped service account requires cluster admin to work with persistent volumes in gke. I would really like to find some docs on this. (other than this thread) – Aric Feb 08 '21 at 18:09