Stack Overflow
Stack Overflow

Questions tagged [gsutil]

gsutil is a command-line program that allows you to store, share and manage data hosted at cloud-based storage providers.

gsutil allows you to store, share and manage data hosted at cloud-based storage providers. It provides a simple Unix shell command-like syntax, operating over buckets and objects named with a URIs and wildcards.

868 questions
0
votes
1 answer

Copy data from google datastore to CSV

I am storing data in google cloud datastore via an appengine project and would like to download one of the entities as a CSV file. I have set up gsutil so that it defaults to this appengine project. I have also created a bucket under cloud…
Nava
  • 71
  • 1
  • 4
0
votes
3 answers

How do I get a refresh token for command line gsutil to work?

I use gsutil to transfer files from a Windows machine to Google Cloud Storage. I have not used it for more than 6 months and now when I try it I get: Failure: invalid_grant From researching this I suspect the access token is no longer valid as it…
user3610488
  • 81
  • 1
  • 2
  • 6
0
votes
2 answers

GSUtil cp moethod corrupt files

In the last week I have noticed that about 5-10% of the gz files I copy down using GSUtils (now v3.42) are corrupt... When I look at the files from the GCS UI, sure enough they are larger than what GSUtils downloaded. (FileNameHere).gz: Unexpected…
Mark
  • 538
  • 6
  • 17
0
votes
1 answer

Gsutil setting All Authenticated Users permission

My google app engine application refers images in a storage bucket. I uploaded them using the Developer Console. Right now when I load my application and error occurs. Fatal error: Uncaught exception…
A.M.N.Bandara
  • 1,490
  • 15
  • 32
0
votes
0 answers

Automatic Parallel Composite Uploads on GAE for Python

I want to upload the file using the Automatic Parallel Composite Uploads of gsutil to Cloud Storage from Google App Engine. That I want you to tell me there are two. The contents of the above whether it is possible. If possible, it is insufficient…
Yusuke Fukui
  • 1
0
votes
0 answers

Retrying failed copies using gsutil cp on Windows

I'm using gsutil on Windows. Sometimes, when copying files from my PC to the Google Cloud Storage bucket I will get messages such as: "CommandException: Some temporary components were not uploaded successfully. Please retry this upload"…
user2980216
  • 41
  • 2
0
votes
0 answers

gsutil authentication code failure

I need to download files from GCS to local machine. I tried gsutil config and download in my machine. In my win 8 64 bit, all worked fine. However when I try to setup the same in another dedicated machine which is win vista 23 bit, entering…
Dhurka
  • 31
  • 2
0
votes
1 answer

Google Cloud Storage Client Library - Check Shared Publicly By Default

I am using Google cloud storage client library for uploading files into it dynamically and serving the files from it. The uploading part is working fine. There is no problem in that. The issue i am facing is after upload if I try to access the file…
Bharath
  • 11
  • 1
0
votes
1 answer

What does gsutil cp do before it begins transferring?

When you run gsutil cp, it will usually say something like: Copying s3://your_bucket/your_file [Content-Type=binary/octet-stream]... At that time the CPU becomes busy with the process for a while before its weight is put on the network. My question…
Seb
  • 152
  • 1
  • 10
0
votes
1 answer

google storage bucket upload permissions

I have tried to create "vendor dropbox" , using the instructions in Google cloud storage documntation the folloing set of commands were executed : creating the bucket gsutil mb gs://customer-10 adding external user permissions gsutil chacl…
user2135521
  • 81
  • 4
0
votes
0 answers

gsutil Cerfificate failure

I recently updated to lates GAE SDK. Gsutil started not working, so I deleted cacerts.txt, as suggested on another post here, but it did not work. Failure: CA certificates file does not have expected SHA1 sum; expected:ed0... actual: da39a3ee5e6...
Emmet B
  • 5,341
  • 6
  • 34
  • 47
0
votes
1 answer

Download using gsutil

I was using gsutil to download a trace file from google storage. The command I used was: gsutil/gsutil cp gs://clusterdata-2011-1/task_usage/part-00499-of-00500.csv.gz ./ But I got an error: GSReponseError: Status=404, code=NoSuchKey, reason=Not…
user1687035
  • 253
  • 1
  • 4
  • 13
-1
votes
1 answer

Issue with Work Identity Federation (WIF) and gsutil ACL set Command

I've an existing setup in which I'm using workload identity federation (WIF) to authenticate circleCI with GCP and everything has been working perfectly fine. Its a simple workflow which uses gsutil -m rsync -d -r folder/ gs://bucket command to sync…
Saif
  • 1,745
  • 5
  • 23
  • 46
-1
votes
2 answers

Move files within same GCP storage bucket and keep version history

How do you move files within the same storage bucket to different folders while keeping their version history? I couldn't find any related option for gsutil mv/cp.
Andi
  • 650
  • 2
  • 8
  • 22
-1
votes
1 answer

gsutil hangs on large file

I'm uploading a file that is 8.6T in size. $ nohup gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp big_file.jsonl gs://bucket/big_file.jsonl > nohup.mv-big-file.out 2>&1 & At some point, it just hangs, with no error messages,…
Bob van Luijt
  • 7,153
  • 12
  • 58
  • 101
1 2 3
…
57
58