Stack Overflow
Stack Overflow

Questions tagged [gsutil]

gsutil is a command-line program that allows you to store, share and manage data hosted at cloud-based storage providers.

gsutil allows you to store, share and manage data hosted at cloud-based storage providers. It provides a simple Unix shell command-like syntax, operating over buckets and objects named with a URIs and wildcards.

868 questions
11
votes
4 answers

Mass rename objects on Google Cloud Storage

Is it possible to mass rename objects on Google Cloud Storage using gsutil (or some other tool)? I am trying to figure out a way to rename a bunch of images from *.JPG to *.jpg.
joshhunt
  • 5,197
  • 4
  • 37
  • 60
10
votes
5 answers

why am i getting (gsutil): "C:\Users\user\AppData\Local\Programs\Python\Python37\python.exe": command not found

After installing Google cloud sdk and connecting to desired firebase project i am receiving : ERROR: (gsutil) "C:\Users\user\AppData\Local\Programs\Python\Python37\python.exe": command not found when running any gsutil command. My current stup…
caffieineAnxiety
  • 101
  • 1
  • 3
10
votes
6 answers

Is there a way to grep through text documents stored in Google Cloud Storage?

Question Is there a way to grep through the text documents stored in Google Cloud Storage? Background I am storing over 10 thousand documents (txt file) on a VM and is using up space. And before it reaches the limit I want to move the documents to…
tetsushi awano
  • 143
  • 1
  • 1
  • 5
10
votes
2 answers

gsutil cannot copy to s3 due to authentication

I need to copy many (1000+) files to s3 from GCS to leverage an AWS lambda function. I have edited ~/.boto.cfg and commented out the 2 aws authentication parameters but a simple gsutil ls s3://mybucket fails from either an GCE or EC2 VM. Error is…
yan-hic
  • 1,374
  • 11
  • 26
10
votes
3 answers

Google cloud storage - Download file from web

I want to use Google cloud storage in my next project. My aim is tracking various web sites and collecting some photos. As, I read the documentation for gsutil; I'm able download the file manually to my server and upload it google cloud storage by…
fobus
  • 1,938
  • 8
  • 29
  • 48
9
votes
3 answers

`gsutil cp` command throwing "OSError: The filename, directory name, or volume label syntax is incorrect"

I've followed the gsutil install guide following the Google Cloud instructions and I've updated GCloud components to the latest versions. I've just recently installed and configured GCloud to work with my credentials and project. First, I've used…
Metafaniel
  • 29,318
  • 8
  • 40
  • 67
9
votes
2 answers

How to pipe input to gsutil cp from stdin?

I am looking for a way to pipe the data into gsutil cp: echo 'foobar' | gsutil cp --stdin gs://my-bucket/foo the -I option is for reading filepaths, not the data..gsutil help cp…
user12211419
9
votes
5 answers

gsutil no longer works?

I've been using gcloud and gsutil for a while but now suddenly for any gsutil command I run I get errors: Traceback (most recent call last): File "/Users/julian/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 12, in import bootstrapping …
julian
  • 368
  • 3
  • 12
9
votes
5 answers

Change storage class of (existing) objects in Google Cloud Storage

I recently learnt of the new storage tiers and reduced prices announced on the Google Cloud Storage platform/service. So I wanted to change the default storage class for one of my buckets from Durable Reduced Availability to Coldline, as that is…
fynali
  • 373
  • 3
  • 12
9
votes
2 answers

gsutil rsync with gzip compression

I'm hosting publicly available static resources in a google storage bucket, and I want to use the gsutil rsync command to sync our local version to the bucket, saving bandwidth and time. Part of our build process is to pre-gzip these resources, but…
regretoverflow
  • 2,093
  • 1
  • 23
  • 45
9
votes
1 answer

How to perform logging with gsutil rsync

What's the proper way to log any errors or warnings when performing a quiet rsync? This is what I currently run from my crontab: gsutil -m -q rsync -r -C /mount1/share/folder gs://my-bucket-1/folder/ > /mount2/share/folder/gsutil.log Since the log…
fredrik
  • 9,631
  • 16
  • 72
  • 132
9
votes
2 answers

gsutil results in "CommandException: You have multiple types of configured credentials"

I'm trying to run a gsutil command to list the contents within a bucket on Google Cloud Storage. I'm almost certain there's nothing wrong with the command. The error I'm getting is: CommandException: You have multiple types of configured…
scott_gl3
  • 93
  • 1
  • 3
8
votes
0 answers

Download individual ARN reports from Google Play

I am in need of downloading reports of individual instances of ANRs (application not responding) data. This in order to upload such data to an internal system that would help me generate reports that are not available on Google Play (ANR rate,…
user3745751
  • 121
  • 5
8
votes
2 answers

Is there a way to use gsutil while impersonating a service account?

I am in the process of attempting to adjust user permissions in Google Cloud and have created a service account that other users can impersonate to access various projects. The gcloud command has the --impersonate-service-account option to make API…
bboe
  • 4,092
  • 3
  • 29
  • 39
8
votes
3 answers

How to get gsutil to use gcloud credentials in a docker container

Why doesn't gsutil use the Gcloud credentials as it should when running in a docker container on Cloud Shell? According to [1] gsutil should use gcloud credentials when they are available: Once credentials have been configured via gcloud auth,…
KevH
  • 479
  • 5
  • 8
1 2
3
…
57 58