Stack Overflow
Stack Overflow

Questions tagged [gcsfuse]

gcsfuse is a user-space file system for interacting with Google Cloud Storage.

149 questions
3
votes
1 answer

Google Cloud Storage FUSE - Using gcsfuse fills up local instance memory

I've been using gcsfuse (FUSE) for some weeks and everything was running smoothly until my instance disk(10GB) got filled up out of nowhere. I was trying to identify the cause and erasing some temporal files and found out that unmounting the bucket…
adrastious
  • 63
  • 8
3
votes
1 answer

Google cloud storage bucket is getting unmounted

I am running some application on google compute engine where it the application reads the data from google cloud storage and writes data to persistent disk. And the bucket is mounted using gcsfuse. But in the middle the bucket is getting unmounted…
Santhosh
  • 91
  • 1
  • 4
3
votes
1 answer

Data written with gsutil is not visible with gcsfuse

I have installed gcsfuse to support an app requiring a posix-like mount point. Existing data written with gsutil is not visible, but data written via the browser (Cloud Storage > Storage Browser) is. According to…
fish
  • 31
  • 1
  • 2
2
votes
0 answers

gcsfuse mount exits with status 1 with the right permisions

I'm trying to mount my database to my container using gcsfuse but im getting this error when i run gcsfuse storage-name /media i get this error: ... 2022/08/07 20:33:42.908030 Mounting file system "development-videoo-storage1"... daemonize.Run:…
Turgut
  • 711
  • 3
  • 25
2
votes
1 answer

Slurm cluster in Google cloud: Data in mounted directory in controller/login node not available in compute nodes

I have created a slurm cluster following this tutorial. I have also created a data bucket that stores some data that needs to be accessed in the compute nodes. Since the compute nodes share the home directory of the login node, I mounted the bucket…
jubueche
  • 763
  • 5
  • 24
2
votes
1 answer

GCSfuse "Unknown bucket" when Gcloud can see it. Service account seems correct

I'm trying to mount a bucket to my gcp instance, but it fails both with a newly created bucket and my existing one. I can see the buckets with gsutil ls, and I can copy files to them with gsutil cp. However, when I try to mount using GCSFuse, the…
LearnDude
  • 83
  • 8
2
votes
0 answers

Terraform: automatically authenticate to mount GCP bucket to VM

I'm able to use Terraform to launch a VM, and if I shell to that VM from the GCP web UI, I can manually authenticate with: gcloud auth application-default login gcloud auth login and then I can manually mount a GCP bucket to the VM: gcsfuse…
Keith Wiley
  • 683
  • 2
  • 6
  • 14
2
votes
1 answer

How to wait for gcsfuse to write-through (flush) to the GCS storage bucket?

After a Compute Engine worker node writes files into a gcsfuse mounted local directory and closes them, I want it to synchronously flush the data through to GCS before it notifies other worker nodes that all the files are ready. This is to ensure…
Jerry101
  • 12,157
  • 5
  • 44
  • 63
2
votes
2 answers

Problem running gcsfuse on Google App Engine

I am trying to run Airflow Webserver on App Engine Flexible however for it to work I need a mounted GCS bucket. I am using custom runtime. The reason why I am doing it is to get a secured endpoint that app Engine provides together with IAP. My…
Tomasz
  • 658
  • 1
  • 7
  • 22
2
votes
1 answer

Allow Google Cloud Compute Engine Instance to write file to Google Storage Bucket - Python

In my python server script which is running on a google cloud VM instance, it tries to save an image(jpeg) in the storage. But it throws following error. File "/home/thamindudj_16/server/object_detection/object_detector.py", line 109, in detect…
Thamindu DJ
  • 1,679
  • 4
  • 16
  • 29
2
votes
1 answer

Why can GCSfuse not see a GS bucket, when GSutil can?

I am working on a Google Compute instance (Ubuntu), and have previously had success using the GCSfuse utility to mount a storage bucket to a local mount point. Today I went to do so again, and have been greeted with the following output: Using mount…
Cam
  • 2,026
  • 3
  • 25
  • 42
2
votes
1 answer

Trouble installing fuse on Debian stretch Docker image

I am attempting to fuse a dir in a Docker image using gcsfuse. I am using a Debian stretch image, and having trouble working with the fuse package. I have attempted to install fuse both via apt-get as well as build from the source via the git repo.…
walking_the_cow
  • 461
  • 4
  • 16
2
votes
0 answers

Slow performance after syncing storage bucket with gcsfuse

I've run into a bug that seemingly has no explanation as to why its happening. Running scripts that are in my mounted drive(mounted using gcsfuse) takes forever to run(almost a couple of minutes per command). However, any scripts that I run from…
Aswin Gopal
  • 21
  • 1
2
votes
1 answer

Google Storage Bucket contents copied from another bucket with "gsutil cp" doesn't show up when the bucket is mounted

I have an "old" Google Storage Bucket, which I'll call "foo-test" here, and a "new" bucket, "foo-01" (not their real names). I was able to copy the contents of the old bucket into the new bucket using "gsutil cp". It worked. Or so I thought. If I go…
hbquikcomjamesl
  • 111
  • 11
2
votes
1 answer

Mount Bucket on Google Storage

I want to mount a Google bucket to a local server. However, when I run the line, the directory I point it to is empty. Any ideas? gcsfuse mssng_vcf_files ./mountbucket/ It reports: File system has been successfully mounted. but the directory…
James Kozubek
  • 21
  • 4
1 2
3
…
9 10