0

I have Google Cloud projects A, B, C, D. They all use similar setup for Kubernetes cluster and deployment. Projects A,B and C have been build months ago. They all use Google Cloud SQL proxy to connect to Google Cloud SQL service. Now when recently I started setting up the Kubernetes for project D, I get following error visible in the Stackdriver logging:

the default Compute Engine service account is not configured with sufficient permissions to access the Cloud SQL API from this VM. Please create a new VM with Cloud SQL access (scope) enabled under "Identity and API access". Alternatively, create a new "service account key" and specify it using the -credential_file parameter

I have compared the difference between the Kubernetes cluster between A,B,C and D but they appear to be same.

Here is the deployment I am using

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: my-site
  labels:
    system: projectA
spec:
  selector:
    matchLabels:
      system: projectA
  template:
    metadata:
      labels:
        system: projectA
    spec:
      containers:
        - name: web
          image: gcr.io/customerA/projectA:alpha1
          ports:
            - containerPort: 80
          env:
            - name: DB_HOST
              value: 127.0.0.1:3306
            # These secrets are required to start the pod.
            # [START cloudsql_secrets]
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: cloudsql-db-credentials
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: cloudsql-db-credentials
                  key: password
          # [END cloudsql_secrets]
        # Change <INSTANCE_CONNECTION_NAME> here to include your GCP
        # project, the region of your Cloud SQL instance and the name
        # of your Cloud SQL instance. The format is
        # $PROJECT:$REGION:$INSTANCE
        # [START proxy_container]
        - name: cloudsql-proxy
          image: gcr.io/cloudsql-docker/gce-proxy:1.11
          command:
            - sh
            - -c
            - /cloud_sql_proxy -instances=my-gcloud-project:europe-west1:databaseName=tcp:3306
            - -credential_file=/secrets/cloudsql/credentials.json
          # [START cloudsql_security_context]
          securityContext:
            runAsUser: 2  # non-root user
            allowPrivilegeEscalation: false
          # [END cloudsql_security_context]
          volumeMounts:
            - name: cloudsql-instance-credentials
              mountPath: /secrets/cloudsql
              readOnly: true
      # [END proxy_container]
      # [START volumes]
      volumes:
        - name: cloudsql-instance-credentials
          secret:
            secretName: cloudsql-instance-credentials
      # [END volumes]

So it would appear that the default service account doesn't have enough permissions? Google Cloud doesn't allow enabling the Cloud SQL API when creating the cluster via Google Cloud console.

From what I have googled this issue some say that the problem was with the gcr.io/cloudsql-docker/gce-proxy image but I have tried newer versions but the same error still occurs.

Camoflame
  • 84
  • 1
  • 4

1 Answers1

0

I found solution to this problem and it was setting the service-account argument when creating the cluster. Note that I haven't tested what are the minimum required permissions for the new service account.

Here are the steps:

  • Create new service account, doesn't require API key. Name I used was "super-service"
  • Assign roles Cloud SQL admin, Compute Admin, Kubernetes Engine Admin, Editor to the new service account
  • Use gcloudto create the cluster like this using the new service account
gcloud container clusters create my-cluster \
--zone=europe-west1-c \
--labels=system=projectA \
--num-nodes=3 \
--enable-master-authorized-networks \
--enable-network-policy \
--enable-ip-alias \
--service-account=super-service@project-D.iam.gserviceaccount.com \
--master-authorized-networks <list-of-my-ips>

Then the cluster and the deployment at least was deployed without errors.

Camoflame
  • 84
  • 1
  • 4
  • 2
    Your answer is correct, but you could have also added the required roles to the Compute Engine Default Service Account. – John Hanley Nov 27 '19 at 14:41
  • @JohnHanley , Initially I did try adding the "Cloud SQL Admin" role to default service account but that was not enough. Thats why it was bit confusing for me. – Camoflame Nov 28 '19 at 07:15
  • @JohnHanley “This default service account may or may not have permissions to use the Google Cloud services you need. It is possible to expand the scopes for the default service account, but that can create security risks and is not recommended.” (https://cloud.google.com/kubernetes-engine/docs/tutorials/authenticating-to-cloud-platform) – Quico Moya Oct 16 '20 at 07:14
  • @QuicoMoya - There are many facets to selecting a strategy for service account (credential/secret) management. If the default service account does not have the required roles, then another service account must be created and then the key material must be distributed to the container(s). That, in my opinion, is also a security risk. Balancing the IAM roles that the cluster requires and the individual containers, requires planning. My comment is not what is best, but what is possible. – John Hanley Oct 16 '20 at 14:10