In Cluster Config is unable to get pods when deployed in a non-default namespace

Question

When I deploy my golang service to any namespace but the default namespace, the service is unable to retrieve pods on any namespace. The same service deployed on the default namespace works perfectly, using the golang client-go api.

Is this a security issue?

Thanks.

did you check whether or not you service has permission? – nightfury1204 Nov 20 '18 at 14:13 — nightfury1204, Nov 20 '18 at 14:13
@nightfury1204 Not sure- where do I check that? – gkgkgkgk Nov 20 '18 at 14:19 — gkgkgkgk, Nov 20 '18 at 14:19
how did you create the client in your golang service? – nightfury1204 Nov 20 '18 at 14:29 — nightfury1204, Nov 20 '18 at 14:29
@nightfury1204 I used rest.InClusterConfig(config) – gkgkgkgk Nov 20 '18 at 14:31 — gkgkgkgk, Nov 20 '18 at 14:31

score 3 · Accepted Answer · answered Nov 20 '18 at 14:53

This issue is permission issue. Since you are using rest.InClusterConfig(config) to create client. That means it using pod's service account as credential. So check whether that service account has the permission to get pods in any namespace.

if service account in the pod is not defined, then it will use default service account.

If RBAC is enabled in your cluster, then check the role binding in that namespace, to find out whether your service account has the permission.

# to see the list of role bindings in 'default' namespace
kubectl get rolebindings --namespace default

To see the specific rolebinding

kubectl get rolebindings ROLE-BINDING-NAME --namespace default -o yaml

Also you can create role and role binding to give permission. To know about RBAC role and role binding see here: https://kubernetes.io/docs/reference/access-authn-authz/rbac/

score 2 · Answer 2 · edited Mar 27 '23 at 17:43

Following is what I used on a minikube cluster to give the default service account access to crud ops on common resources. The obvious caveat is that you'd need to be careful on a real cluster.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: crud-role
  namespace: default
rules:
- apiGroups: ["", "apps", "batch"]
  resources: [ "deployments", "jobs", "pods", "replicasets", "services" ]
  verbs: [ "create", "get", "list", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: crud-role-binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: crud-role
subjects:
  - kind: ServiceAccount
    name: default
    namespace: default

score 0 · Answer 3 · answered Mar 27 '23 at 17:49

I was getting a similar error but from a pod using golang client in default namespace:

pods is forbidden: User "system:serviceaccount:default:default" cannot list resource "pods" in API group "" at the cluster scope

Golang code snippet:

if configMode == "IN_CLUSTER" {
    // creates the in-cluster config
    config, err := rest.InClusterConfig()
    if err != nil {
        panic(err.Error())
    }
    return config, err
}

I used the solution provided by @Gazi but with ClusterRole, that will let you access resources cluster-wide. It was modified for get and list only:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: query-role
  namespace: default
rules:
- apiGroups: ["", "apps", "batch"]
  resources: [ "deployments", "jobs", "pods", "replicasets", "services" ]
  verbs: [ "get", "list" ]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: query-role-binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: query-role
subjects:
  - kind: ServiceAccount
    name: default
    namespace: default

Following links are useful: https://kubernetes.io/docs/reference/access-authn-authz/rbac/ https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

In Cluster Config is unable to get pods when deployed in a non-default namespace

3 Answers3

Linked