0

I am using the cloud.google.com/go SDK to programmatically provision the GKE clusters with the required configuration.

I set the ClientCertificateConfig.IssueClientCertificate = true (see https://pkg.go.dev/google.golang.org/genproto/googleapis/container/v1?tab=doc#ClientCertificateConfig). After the cluster is provisioned, I use the ca_certificate, client_key, client_secret returned for the same cluster (see https://pkg.go.dev/google.golang.org/genproto/googleapis/container/v1?tab=doc#MasterAuth). Now that I have the above 3 attributes, I try to generate the kubeconfig for this cluster (to be later used by helm)

Roughly, my kubeconfig looks something like this:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <base64_encoded_data>
    server: https://X.X.X.X
  name: gke_<project>_<location>_<name>
contexts:
- context:
    cluster: gke_<project>_<location>_<name>
    user: gke_<project>_<location>_<name>
  name: gke_<project>_<location>_<name>
current-context: gke_<project>_<location>_<name>
kind: Config
preferences: {}
users:
- name: gke_<project>_<location>_<name>
  user:
    client-certificate-data: <base64_encoded_data>
    client-key-data: <base64_encoded_data>

On running kubectl get nodes with above config I get the error: Error from server (Forbidden): serviceaccounts is forbidden: User "client" cannot list resource "serviceaccounts" in API group "" at the cluster scope

Interestingly if I use the config generated by gcloud, the only change is in the user section: user:

auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: /Users/ishankhare/google-cloud-sdk/bin/gcloud
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp

This configuration seems to work just fine. But as soon as I add client cert and client key data to it, it breaks: user:

auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: /Users/ishankhare/google-cloud-sdk/bin/gcloud
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp
client-certificate-data: <base64_encoded_data>
client-key-data: <base64_encoded_data>

I believe I'm missing some details related to RBAC but I'm not sure what. Will you be able to provide me with some info here?

Also reffering to this question I've tried to only rely on Username - Password combination first, using that to apply a new clusterrolebinding in the cluster. But I'm unable to use just the username password approach. I get the following error:

error: You must be logged in to the server (Unauthorized)
Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
Ishan Khare
  • 1,745
  • 4
  • 28
  • 59
  • What version of K8s you want to use? Did you follow any tutorial? Are you doing everything from ClouShell? – PjoterS Mar 30 '20 at 18:39
  • There seems to be a different set of configurations. One generated with the SDK that doesn't have the clusterrole necessary and the one generated by gcloud that has the clusterrolebinding with the appropiate clusterrole. How are you generating the first one? – PjoterS Apr 03 '20 at 14:28

0 Answers0