Stack Overflow
Stack Overflow

Questions tagged [gsutil]

gsutil is a command-line program that allows you to store, share and manage data hosted at cloud-based storage providers.

gsutil allows you to store, share and manage data hosted at cloud-based storage providers. It provides a simple Unix shell command-like syntax, operating over buckets and objects named with a URIs and wildcards.

868 questions
16
votes
1 answer

Can you pass a keyfile.json to gsutil?

I have a (maybe unique?) use case in some Python scripts that I am running. Namely, I want the parallel awesomeness of gsutil and so I don't do from google.cloud import storage, rather I use subprocess calls such as: subprocess.Popen(["gsutil",…
shogekiha
  • 175
  • 1
  • 5
16
votes
8 answers

Cloud Storage - No 'Access-Control-Allow-Origin' header is present on the requested resource for AngularJS view

According to Chrome dev tools, my requests to get my html partials have the origin header https://site-name-here.com and request header GET. I have the following JSON file set to my bucket: [ { "origin": ["https://site-name-here.com"], …
James Walker
  • 393
  • 1
  • 3
  • 11
15
votes
3 answers

Authorize Google Cloud Platform Service Account to Access Only One Google Cloud Storage Bucket

I have created a Google Cloud Platform service account, $GCP_SERVICE_ACCOUNT, with the Storage Admin (roles/storage.admin) role. Now I wish to restrict this account so that it can only access a specific Google Cloud Storage (GCS) Bucket…
i_am_cris
  • 557
  • 1
  • 5
  • 19
15
votes
4 answers

How to retrieve the most recent file in cloud storage bucket?

Is this something that can be done with gsutil? https://cloud.google.com/storage/docs/gsutil/commands/ls does not seem to mention any sorting functionality - only filtering by a date - which wouldn't work for my use case.
Chris Stryczynski
  • 30,145
  • 48
  • 175
  • 286
15
votes
3 answers

ImportError: No module named google_compute_engine

I used google compute engine from Google Cloud Platform , and I want to store my data in googles storage, so I used the gsutil to send data from compute engine to google storage Before i updated python2.6.6 to python2.7, gsutil was working…
Shiue-yang Shiau
  • 171
  • 1
  • 1
  • 6
15
votes
7 answers

Moving multiple files with gsutil

Let's say I've got the following files in a Google Cloud Storage bucket: file_A1.csv file_B2.csv file_C3.csv Now I want to move a subset of these files, lets say file_A1.csv and file_B2.csv. Currently I do this like that: gsutil mv…
toom
  • 12,864
  • 27
  • 89
  • 128
14
votes
2 answers

How to run Google gsutil using Python

After installing and configuring Google Cloud SDK gsutil command can be run by simply typing its name and the argument(-s) using Windows cmd. Here is the example: "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin\gcloud" version But the…
alphanumeric
  • 17,967
  • 64
  • 244
  • 392
14
votes
2 answers

Google Cloud Storage ACL confusion

I'm the owner of a Google Cloud project, with a Google Cloud Storage bucket inside. All our backups are moved to this bucket. When I try to retrieve some of the backups, I get a permission denied. I'm not able to do anything but to list the…
Hedam
  • 2,209
  • 27
  • 53
13
votes
2 answers

What should I do about this gsutil "parallel composite upload" warning?

I am running a python script and using the os library to execute a gsutil command, which is typically executed in the command prompt on Windows. I have some file on my local computer and I want to put it into a Google Bucket so I do: import…
user1367204
  • 4,549
  • 10
  • 49
  • 78
13
votes
3 answers

Handle gsutil ls and rm command errors if no files present

I am running the following command to remove files from a gcs bucket prior to loading new files there. gsutil -m rm gs://mybucket/subbucket/* If there are no files in the bucket, it throws the "CommandException: One or more URLs matched no…
user1311888
  • 773
  • 3
  • 11
  • 24
13
votes
7 answers

gsutil make bucket command [gsutil mb] is not working

I am trying to create a bucket using gsutil mb command: gsutil mb -c DRA -l US-CENTRAL1 gs://some-bucket-to-my-gs But I am getting this error message: Creating gs://some-bucket-to-my-gs/... BadRequestException: 400 Invalid argument. I am following…
K M Rakibul Islam
  • 33,760
  • 12
  • 89
  • 110
12
votes
3 answers

google-cloud-sdk installation fails on python syntax error

on a "Linux supermicro 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux" I see this happening when doing "sudo apt-get install google-cloud-sdk" > Reading package lists... Done Building dependency tree …
Karel Goderis
  • 163
  • 1
  • 5
11
votes
5 answers

Error "No URLs matched" When copying Google cloud bucket data to my local computer?

I am trying to download a folder which is inside my Google Cloud Bucket, I read from google docs gsutil/commands/cp and executed below the line. gsutil cp -r appengine.googleapis.com gs://my-bucket But i am getting the error CommandException: No…
Sudhanshu Gaur
  • 7,486
  • 9
  • 47
  • 94
11
votes
3 answers

gsutil config -a. Which key to use?

There are so many keys to be created for use with Google Compute Cloud. I'm trying to create .boto file using gsutil and it's asking for "Google access key ID". My goal is to script access to "Google Cloud Storage" using Python. So, when gsutil is…
Alex B
  • 2,165
  • 2
  • 27
  • 37
11
votes
1 answer

Why is gsutil -m cp -R much faster than gsutil cp -R

Hi all I am using GCE and now I'm new to bucket. I have tons of pics on my other bucket. I tried copying it as sudo gsutil cp -R dir dir it took me 30 mins still didn't finished the file copy so I tried the suggestion which is to use the -m…
Emil Reña Enriquez
  • 2,929
  • 1
  • 29
  • 32
1
2
3
…
57 58