1

I'm having trouble when uploading folders onto my google-cloud-storage bucket: They wont show up in the mounted drive /mnt/willferbucket (with gcsfuse)

It doesnt matter if i upload them through the webfrontend or by using gsutil, the only folders that show up, are the ones i create (not upload!, doesnt matter via webfrontend or directly on the mounted drive, in the example below: "canvas" and "prints", so this is working)

So.. "ls" on the mounted drive looks like this:

root@ubuntu-2:/mnt/willferbucket# ls
canvas  helloWorld.py  helloWorldSimple.py  prints  test.txt

But as you can see, when using gsutil:

There are my uploaded folders showing up and i'm able to download (same in the webfrontend: they show up):

root@ubuntu-2:/mnt/w# gsutil ls gs://willferbucket
gs://willferbucket/helloWorld.py
gs://willferbucket/helloWorldSimple.py
gs://willferbucket/test.txt
gs://willferbucket/canvas/
gs://willferbucket/prints/
gs://willferbucket/test/
gs://willferbucket/testfolder/
gs://willferbucket/tst/

I couldn't find out what the reason for this behaviour is :( Maybe someone can help or is facing the same problem

Thanks for your reply

Andrew Gaul
  • 2,296
  • 1
  • 12
  • 19
Mark Foster
  • 11
  • 2
  • I guess the user who mounted the file system might not have the viewing permission in your existing "gs://willferbucket". Can you make sure the user has such a permission for the bucket? https://github.com/GoogleCloudPlatform/gcsfuse/blob/master/docs/mounting.md – greeness May 23 '18 at 00:07

1 Answers1

7

I have done the same test as you, using gcsfuse to mount one of my Cloud Storage buckets into my local system, and it also appears incomplete to me.

I have notified this situation to the Google Cloud Storage engineering team on your behalf and the issue is currently being handled by them. You can keep track on any progress they have made by following this link.

Please click on the star button next to the issue number to get email notifications on how the situation is being handled.

UPDATE

The Google Cloud Storage engineering team has come back to me and pointed out that the reason not all files and directories are listed when using gcsfuse is because of what would be called implicit directories, which are not set by just using gcsfuse (like an mdkir operation inside the mounted bucket), but through other means such as the Cloud Storage console. These directories may not be recognized by gcsfuse and therefore not added to the mounted bucket in your file system.

There is a way to solve this. Whenever you run the gcsfuse command to mount a Cloud Storage bucket into your local file system, add the --implicit-dirs flag so that all implicit directories are included. This would be an example for it:

gcsfuse --implicit-flags [YOUR_BUCKET] /path/to/your/local/dir/
Rodrigo C.
  • 1,114
  • 7
  • 13