2

I'm using a juicefs-csi in GKE. I use postgre as meta-store and GCS as storage. The corresponding setting is as follow:

node:
  # ...
  storageClasses:
    - name: juicefs-sc
      enabled: true
      reclaimPolicy: Retain
      backend:
        name: juicefs
        metaurl: postgres://user:password@my-ec2-where-postgre-installed.ap-southeast-1.compute.amazonaws.com:5432/the-database?sslmode=disable
        storage: gs
        bucket: gs://my-bucket
   # ...

According to this documentation, I don't have to specify access key/secret (like in S3).

But unfortunately, whenever I try to write anything to the mounted volume (with juicefs-sc storage class), I always get this error:

AccessDeniedException: 403 Caller does not have storage.objects.create access to the Google Cloud Storage object.

I believe it should be related to IAM role.

My question is, how could I know which IAM user/service account is used by juicefs to access GCS, so that I can assign a sufficient role to it?

Thanks in advance.

EDIT

Step by step:

  • Download juicefs-csi helm chart
  • Add values as described in the question, apply
  • Create a pod that mount from PV with juicefs-sc storage class
  • Try to read/write file to the mount point
Wytrzymały Wiktor
  • 11,492
  • 5
  • 29
  • 37
goFrendiAsgard
  • 4,016
  • 8
  • 38
  • 64
  • Could you elaborate your actions/steps? You mention GCP but in metaurl you have aws. In GCP most popular autohorization option is to use proper `ServiceAccount`, more details [here](https://cloud.google.com/docs/authentication/getting-started). Also in your docs you have link to options how to auth your request [here](https://cloud.google.com/docs/authentication#examples). If you didn't create it means you are using default GCP SA which permissions depends on your project. You probably need to create SA with `Storage Object Creator` role. – PjoterS Dec 14 '21 at 14:04
  • One way to know the SA, would be to go to MENU> LOGGING> look for the permission denied error, expand the error and if you look for the principal email that should give you an indication of which SA is failing. – dany L Dec 14 '21 at 19:30
  • Hi @PjoterS sorry for miss-lead. My postgre db is hosted in ec2. Juicefs need "meta URL" to save the configuration/structure and "storage" to save the real object. Thank you for the pointers. – goFrendiAsgard Dec 15 '21 at 01:02
  • Hy @danyL Definitely going to need that. Thank you – goFrendiAsgard Dec 15 '21 at 01:03

1 Answers1

1

Ok I misunderstood you at the beginning.

When you are creating GKE cluster you can specify which GCP Service Account will be used by this cluster, like below:

By Default it's Compute Engine default service account (71025XXXXXX-compute@developer.gserviceaccount.com) which is lack of a few Cloud Product permissions (like Cloud Storage, it has Read Only). It's even described in this message.

If you want to check which Service Account was set by default to VM, you could do this via

Compute Engine > VM Instances > Choose one of the VMs from this cluster > In details find API and identity management

So You have like 3 options to solve this issue:

1. During Cluster creation

In Node Pools > Security, you have Access scopes where you can add some additional permissions.

  • Allow full access to all Cloud APIs to allow access for all listed Cloud APIs
  • Set access for each API

In your case you could just use Set access for each API and change Storage to Full.

2. Set permissions with a Service Account You would need to create a new Service Account and provide proper permissions for Compute Engine and Storage. More details about how to create SA you can find in Creating and managing service accounts.

3. Use Workload Identity

Workload Identity on your Google Kubernetes Engine (GKE) clusters. Workload Identity allows workloads in your GKE clusters to impersonate Identity and Access Management (IAM) service accounts to access Google Cloud services.

For more details you should check Using Workload Identity.

Useful links

PjoterS
  • 12,841
  • 1
  • 22
  • 54