1

I am attempting to mount several Google cloud storage buckets from a Google cloud compute virtual machine instance using gcsfuse. It seems that some of the buckets show their contents correctly and others do not unless the --implicit-dirs options is used in the mount, eg:

gcsfuse --implicit-dirs bucket-name  /mnt/gcs_buckets/bucket-name

Or adding the following to the fstab (/etc/fstab) on the VM:

bucket-name  /mnt/gcs_buckets/bucket-name  gcsfuse  rw,user,allow_other,implicit_dirs

For the buckets where --implicit-dirs is required, the latency for accessing data on the mounted bucket too slow to be useful. I plan to add several entries into the fstab to mount the buckets, and I understand that the difference between the buckets may have something to do with how the objects were added to the bucket.

Mounting it as a file system simplifies the integration requirement for an existing application, so I'd like to first explore whether it is possible to either automatically "fix" the buckets that require the --implicit-dirs argument, or explore other mounting options that don't suffer from the latency problems. If mounting it as a file system is not going to work, then I'll have to go down the route of writing a custom code for the GCS API, but I'd like to explore other options before taking that route.

Does anyone have any insights / advice on whether it is possible to do this as a filesystem mount without suffering from the latency issues? Alternatively is there an easy way of "fixing" the buckets that don't work without the --implicit-dirs argument? I am new to GCS, so any insight/advice would be greatly appreciated.

Ari
  • 31
  • 6
  • The best way is to not use a file system interface such as gcsfuse. Write native GCS code to access GCS. GCS is not a file system but an object storage system. Do not treat it like a file system and you will have much better results. – John Hanley Feb 21 '19 at 21:00
  • Thanks for the prompt reply @JohnHanley. For the buckets that work without `--implicit-dirs`, it appears to work fine. Mounting it as a file system simplifies the integration requirement for my existing application, so I'd like to first explore whether it is possible to either automatically "fix" the buckets that require the `--implicit-dirs` argument, or explore other mounting options that don't suffer from the latency problems. However, if mounting it as a file system is not going to work, then I'll have to go down the route of writing a custom code, but I'd like to avoid that if possible? – Ari Feb 22 '19 at 00:24

1 Answers1

0

An option could be: access this resource, for example imagen.jpg through the public url in the bucket, https://storage.googleapis.com/bucket-name/imagen.jpg, and try to give it the public permissions to the image or resource that you are going to upload in the vm and that automatically through gcsfuse will be saved in your bucket.

Gerson Rubio Garcia
  • 1
  • 1
  • 3