How to pipe input to gsutil cp from stdin?

Question

I am looking for a way to pipe the data into gsutil cp:

echo '<html>foobar</html>' | gsutil cp --stdin gs://my-bucket/foo

the -I option is for reading filepaths, not the data..gsutil help cp says:

https://gist.github.com/ORESoftware/566d9c3d00e3858c1161ae9b741fc07e

So yeah I am looking for a way to send files to GS via stdin.

score 16 · Accepted Answer · answered Jan 24 '20 at 21:44

16

Looks like it works using - like so:

echo '<html>foobar</html>' | gsutil cp - gs://my-bucket/foo

in my opinion, a --stdin flag would be more explicit and preferred.

answered Jan 24 '20 at 21:44

4

Almost everything in Unix that reads from a file, or writes to a file, accept `-` to mean stdin and stdout respectively. This is consistent with half a century of every single command line tool. So that is to say I disagree with you. :-) – Thomas May 25 '21 at 18:46

score -1 · Answer 2 · answered Aug 24 '22 at 08:07

From the docs:

You can use the -I option with stdin to specify a list of URLs to copy, one per line. This allows you to use gsutil in a pipeline to upload or download objects as generated by a program:

cat filelist | gsutil -m cp -I gs://my-bucket

or:

cat filelist | gsutil -m cp -I ./download_dir

where the output of cat filelist is a list of files, cloud URLs, and wildcards of files and cloud URLs.

How to pipe input to gsutil cp from stdin?

2 Answers2