1

I have many folders stored in gcs bucket and i want to delete those folders recursively.

gsutil -m ls -d gs://bucket_name/folder_1/*/ | grep 2021-03-17 | gsutil -m rm -r

So basically i want to delete all the folders recursively whose names contain "2021-03-17" string.

Getting below error when running it:

CommandException: No URLs matched. Do the files you're operating on exist?

I Can't delete each file seperately and my usecase is to delete the folder itself recursively.

How can i solve this error ?

Joseph D
  • 189
  • 1
  • 12

2 Answers2

1

Use xargs to execute gsutil rm -r line per line from the output of grep.

gsutil ls -d gs://bucket_name/folder_1/* | grep 2021-03-17 | xargs gsutil -m rm -r

Test: enter image description here

Ricco D
  • 6,873
  • 1
  • 8
  • 18
0

This sounds like something you'll probably want to write a small wrapper script for. The pipeline you currently have doesn't account for the command output from command #1 being empty. Also, gsutil rm doesn't accept items from stdin unless you use the -I flag (you could also use xargs to substitute the arguments in). I'd say it's worth writing a short wrapper script that does these things:

  • Run the gsutil ls command to fetch the directories (it's possible none exist, which is probably why you're seeing that message).
  • For each directory, see if it matches the regex.
  • Supply the matching paths to gsutil -m rm as arguments. If there are more than ~32k matches (the max number of arguments for a command), you'll need to do them in batches.
mhouglum
  • 2,468
  • 13
  • 21