Stack Overflow
Stack Overflow

Questions tagged [gsutil]

gsutil is a command-line program that allows you to store, share and manage data hosted at cloud-based storage providers.

gsutil allows you to store, share and manage data hosted at cloud-based storage providers. It provides a simple Unix shell command-like syntax, operating over buckets and objects named with a URIs and wildcards.

868 questions
6
votes
2 answers

gsutil CommandException: No URLs matched: *.txt

I'm trying to understand how to copy local files to cloud storage using gsutil so I can write a script to move files. I followed the next steps: C:\Program Files (x86)\Google\Cloud SDK>gsutil ls gs://sa-upload-test/ C:\Program Files…
theosgood
  • 61
  • 1
  • 1
  • 3
6
votes
2 answers

Using gsutil with google drive (not google cloud storage)

gsutil use boto configuration file for authenticate at Google Cloud Storage. I generate ClientID in console.developers.google.com - that allow put files to Google Drive by python script: #!/usr/bin/env python from apiclient.discovery import…
RNZ
  • 61
  • 1
  • 3
6
votes
3 answers

can not authenticate with gcs in python

I am following the example in https://developers.google.com/storage/docs/gspythonlibrary#credentials I created client/secret pair by choosing in the dev. console "create new client id", "installed application", "other". I have the following code in…
Roman
  • 1,351
  • 11
  • 26
6
votes
1 answer

gsutil cp -z how can I copy and gzip multiple file types at once

I have tried gsutil cp -z "js, css, html" file/name gs://bucket/name Only file type that compressed is js but other type remain uncompress. Can cp -z command upload and compress multiple file type? Any suggestion would be appreciate. Thanks in…
sthapaun
  • 310
  • 1
  • 3
  • 8
5
votes
2 answers

Gsutil cp is not copying files

I am trying to use the command line on my local machine (anaconda prompt) to download a folder from a google cloud bucket. I am trying to do so using the gsutil command. I am first able to log in to the project using gcloud auth login. Doing so…
seeker_after_truth
  • 385
  • 3
  • 20
5
votes
3 answers

How to create a empty folder in google storage(bucket) using gsutil command?

How we can create the folder using gsutil command. I am using Bashoperator in airflow where I need to use the gsutil Bash command, Bucket is already created I want to create a folder inside bucket. I already tried with below command but It's not…
Bhagesh Arora
  • 547
  • 2
  • 12
  • 30
5
votes
1 answer

Does gcloud storage python client API support parallel composite upload?

The gsutil command has options to optimize upload/download speed for large files. For example GSUtil:parallel_composite_upload_threshold=150M GSUtil:sliced_object_download_max_components=8 see this page for reference. What is the equivalence in the…
nos
  • 19,875
  • 27
  • 98
  • 134
5
votes
2 answers

Gsutil Not working in Dockerfile on Google Compute Engine

I'm a bit new to Docker and I'm trying to copy resources from my cloud bucket to my instance created with a docker image. I use gsutil with the following in my Dockerfile # Install Google Cloud tools - Debian…
aaronvargas
  • 12,189
  • 3
  • 52
  • 52
5
votes
1 answer

GCE startup script fails to run on Ubuntu 18.04

I am launching an instance on GCP using gcloud compute and specifying the URL of my script using the metadata param. The script fails to run, when I check the metadata, the script url is there. The logs have no mention of any errors. When I sudo -i…
Alex
  • 1,281
  • 1
  • 13
  • 26
5
votes
2 answers

gsutil Failure: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I am getting the below error message while trying to access my google cloud storage from one of my Google Compute Engine instance using gsutil command. Below is the command and error message. Command: gsutil ls gs://my-storage-bucket Error…
Bhasky93
  • 59
  • 1
  • 8
5
votes
2 answers

Copy files from remote server to google cloud bucket

How would I copy files from a remote server to a google bucket? For example, gcloud compute scp username@server:/path/to/file gs://my-bucket This method gives the error: All sources must be local files when destination is remote. Additionally,…
JakeC
  • 292
  • 3
  • 11
5
votes
2 answers

Using standalone 'gsutil' from within GKE

I'm trying to use the standalone gsutil tool from within a container running in a GKE cluster, but I cannot get it to work. I believe the cluster has adequate permissions (see below). However, running ./gsutil ls…
James Howlett
  • 87
  • 1
  • 8
5
votes
1 answer

gsutil rsync exits after starting synchronization

I'm trying to sync folders from Google compute engine instance to cloud storage, but it just exits without any pointers whatsoever. It says Starting synchronization and then goes back to the prompt. gsutil -m rsync -r public/…
Nick Weisser
  • 910
  • 1
  • 9
  • 21
5
votes
2 answers

gsutil zip directory on google cloud storage

is it possible to compress file or directory of google cloud storage without download it first and re-upload? I think I need some tools similar like http://googlegenomics.readthedocs.org/en/latest/use_cases/compress_or_decompress_many_files/ thank…
oon arfiandwi
  • 515
  • 1
  • 8
  • 21
5
votes
2 answers

Abort gsutil transfer

I am transferring a folder containing 100+ heavy files (several hundred MB) gsutil to my google cloud storage (Nearline). I've made a mistake in path of the destination folder. So I am looking for a way to abort the transfer. Here is the command I…
MagTun
  • 5,619
  • 5
  • 63
  • 104
1 2 3
…
57 58