Stack Overflow
Stack Overflow

Questions tagged [gsutil]

gsutil is a command-line program that allows you to store, share and manage data hosted at cloud-based storage providers.

gsutil allows you to store, share and manage data hosted at cloud-based storage providers. It provides a simple Unix shell command-like syntax, operating over buckets and objects named with a URIs and wildcards.

868 questions
7
votes
2 answers

Listing all public links for all objects in a bucket using gsutil

Is there a way to list all public links for all the objects stored into a Google Cloud Storage bucket (or a directory in a bucket) using Cloud SDK's gsutil or gcloud? Something like: $ gsutil ls --public-link gs://my-bucket/a-directory
toomuchkevin
  • 161
  • 2
  • 7
7
votes
2 answers

Why is gsutil rsync re-downloading all our files?

We've been using gsutil -m rsync -r to keep dev and deploy boxes in sync with a GCS bucket for nearly 2 years without any problem. There are about 85k objects in the bucket. Until recently, this worked perfectly: we'd run a deploy-box -> GCS rsync…
Igor Clark
  • 103
  • 1
  • 6
7
votes
3 answers

Google Cloud Storage Bucket creation

I want to create a google bucket if it doesn't exist. Otherwise, I want to reuse the bucket name. How to do it? Its equivalent of the unix command mkdir -p dir_name I used the command but my shell script crashes when I run this next time. gsutil…
nizam.sp
  • 4,002
  • 5
  • 39
  • 63
7
votes
2 answers

Cron - gsutil not found

gsutil has been installed here: /usr/local/bin/gsutil My crontab looks like this (i'm logged in as root): */1 * * * * /home/deploy/cron/job.sh >> /home/deploy/cron/test.log 2>&1 job.sh: #!/bin/sh PATH="$PATH":/usr/local/bin/gsutil now=$(date…
Justin Young
  • 2,393
  • 3
  • 36
  • 62
6
votes
3 answers

How to configure gsutil?

I followed the Google instructions to set up gsutil. It states that after runnig 'gsutil config' in terminal I should see: This script will create a boto config file at /.boto containing your credentials, based on your responses to the following …
qliq
  • 11,695
  • 15
  • 54
  • 66
6
votes
1 answer

Xcode update broke gcloud and gsutil

A few hours ago I updated Xcode to the latest version, which caused gcloud and gsutil commands to stop working. For example, a minute before the update finished, the command gsutil -m rsync was working fine. While writing this question, I noticed I…
ronen
  • 233
  • 1
  • 7
6
votes
1 answer

Authenticating standalone gsutil in containers in Cloud ML Engine on Kubernetes with Workload Identity

I'm launching container images on Google Cloud AI Training (Cloud ML Engine) Inside those containers I need to use gsutil. Some containers have gsutil. In that case I can use it right away without any authentication steps. Some containers do not…
Ark-kun
  • 6,358
  • 2
  • 34
  • 70
6
votes
2 answers

How to copy files from colab or cloud storage to google drive?

I read some code on this problem, but i cannot make sense of the code. Could anybody help explaining the code to me? # mount your drive from google.colab import drive drive.mount('/content/drive') !gsutil -q -m cp -r gs://my-bucket-name drive…
user11366694
  • 141
  • 1
  • 2
  • 8
6
votes
4 answers

List buckets that match a bucket label with gsutil

I have my google cloud storage buckets labeled I can't find anything in the docs on how to do a gsutil ls but only filter buckets with a specific label- is this possible?
red888
  • 27,709
  • 55
  • 204
  • 392
6
votes
2 answers

The mb command requires a URL that specifies a bucket

I'm attempting to use the mb command to create a bucket on Google Cloud Storage but am getting CommandException: The mb command requires a URL that specifies a bucket. The odd part is that while gsutil mb gs://foo/bar1 returns this…
dzy
  • 171
  • 2
  • 8
6
votes
3 answers

GCP file copy from bucket fails with could not reach metadata service

Issue - While using the gsutil command to download a file from google bucket with public access from a GCP instance we get the following error root@openvpn:~# gsutil cp gs://google-artifacts-1/openvpn/openvpn-install.sh…
rohan gupta
  • 139
  • 2
  • 7
6
votes
1 answer

GSUTIL cp file from server to bucket and make file public

Using this command in my server from a php file: exec(gsutil cp /path/to/file/on/server/namefile.ext gs://nameBucket/dir/namefile.ext > /dev/null 2>&1) i got the namefile.ext on bucket, in the correct directory...but need to make this file public…
tidpe
  • 325
  • 3
  • 15
6
votes
2 answers

Is there any advantage to using gsutil or the google cloud storage API in production transfers?

Which is better to use with production transfers, gsutil, or the google cloud storage API?
mohawkTrail
  • 606
  • 1
  • 7
  • 19
6
votes
3 answers

gsutil: Argument list too long

I am trying to upload many thousands of files to Google Cloud Storage, with the following command: gsutil -m cp *.json gs://mybucket/mydir But I get this error: -bash: Argument list too long What is the best way to handle this? I can obviously…
Richard
  • 62,943
  • 126
  • 334
  • 542
6
votes
3 answers

gsutil not working on mac and python3.5

I am trying to use google storage lib but I am getting an error when I call gsutil... Installed gsutil with pip: pip install gsutil my current python version is 3.5 when I run gsutil from command line I get the bellow error: class…
Pippo
  • 905
  • 11
  • 22
1 2 3
…
57 58