How to clean an Azure storage Blob container?

Question

I just want to clean (dump, zap, del .) an Azure Blob container. How can I do that?

Note: The container is used by IIS (running Webrole) logs (wad-iis-logfiles).

Removing and recreating? – Jonatas Walker Nov 22 '22 at 21:45 — Jonatas Walker, Nov 22 '22 at 21:45

joanlofe · Accepted Answer · 2019-01-23T12:03:18.153

38

A one liner using the Azure CLI 2.0:

az storage blob delete-batch --account-name <storage_account_name> --source <container_name>

Substitute <storage_account_name> and <container_name> by the appropriate values in your case.

You can see the help of the command by running:

az storage blob delete-batch -h

edited Jan 23 '19 at 12:03

answered Feb 08 '18 at 16:22

joanlofe

3,360
2
26
44

4

Worked great, after I added `--account-key` – Ruud Helderman May 30 '18 at 18:43
1

In case you see this error: "This operation is not permitted because the blob has snapshots. ErrorCode: SnapshotsPresent", you may want to add this option: `--delete-snapshots include` – Ruud Helderman Mar 15 '19 at 13:29
best answer if you have access to the Azure portal – Rick Oct 29 '19 at 09:35
Worked well. I passed `--connection-string` along with the command. And it should be quotes – Skillz Mar 25 '21 at 17:12
This is really slow for me. Need to find an alternative. – user7788539 Apr 14 '21 at 22:30
for `az storage blob delete-batch --account-name myaccountname --source $web` gives `argument --source/-s: expected one argument` – Piotrek Mar 02 '22 at 21:50

score 26 · Answer 2 · answered May 04 '12 at 05:36

There is only one way to bulk delete blobs and that is by deleting the entire container. As you've said there is a delay between deleting the container and when you can use that container name again.

Your only other choice is to delete the one at a time. If you can do the deleting from the same data centre where the blobs are stored it will be faster than running the delete locally. This probably means writing code (or you could RDP into one of your instances and install cloud explorer). If you're writing code then you can speed up the overall process by deleting the items in parallel. Something similar to this would work:

Parallel.ForEach(myCloudBlobClient.GetContainerReference(myContainerName).ListBlobs(), x => ((CloudBlob) x).Delete());

works very nice - thanks! One correction would be to return files as a flat list in case folders were used: `Parallel.ForEach(blobContainer.ListBlobs(useFlatBlobListing: true), x => ((CloudBlob)x).Delete());` — Eleasar, Feb 10 '17 at 16:36

galdin · Answer 3 · 2018-07-26T04:57:06.467

19

Update: Easier way to do it now (in 2018) is to use the Azure CLI. Check joanlofe's answer :)

Easiest way to do it in 2016 is using Microsoft Azure Storage Explorer IMO.

Download Azure Storage Explorer and install it
Sign in with the appropriate Microsoft Account
Browse to the container you want to empty
Click on the Select All button
Click on the Delete button

edited Jul 26 '18 at 04:57

answered May 04 '16 at 21:08

galdin

12,411
7
56
71

The HOW is very important here - you have to hit load more till you reach thr very last page of blobs and then hit select all. And then delete. – Roberto Bonini Jun 01 '16 at 21:18
1

@RobertoBonini added the steps. Of what I remember I din't have to keep loading till the end. – galdin Jun 03 '16 at 19:56
Select all can be done for "all in page" and "all cached", no need to load all pages. Great tool btw, thanks for sharing this info. – Stefan Cebulak Dec 04 '16 at 13:33
2

Azure Storage Explorer is awesome, but deleting files is dog slow. Try cleaning a container with 100K+ files - it will take forever. – absmiths Mar 26 '18 at 16:26
@absmiths agreed. I paged through 20 pages of 1000 records (just keep hitting page 10, 20, 30, etc), eventually putting 20k records in the cache, which appeared to load just fine. Then I hit "Select All > Cached", but as soon as I clicked Delete it totally locked up my laptop (Core i5 w/8GB RAM) for several minutes until eventually ASE crashed. Repeated the process with the same outcome. Haven't seen a program crash that hard since Win7 days or earlier... – pcdev Jul 26 '18 at 04:48
@pcdev use the CLI :) I'll update my answer to add a link to joanlofe's answer. – galdin Jul 26 '18 at 04:52
Yep, that's the obvious answer, however it would be nice to be able to open a GUI to do a task like this that I probably only do once every 18-24 months rather than have to look up the answer every time :) – pcdev Jul 26 '18 at 04:58

score 6 · Answer 4 · answered May 14 '12 at 13:43

6

Try using cloudberry product for windows azure

this is the link: http://www.cloudberrylab.com/free-microsoft-azure-explorer.aspx

you can search in the blob for specific extension. select multiple blobs and delete them

answered May 14 '12 at 13:43

Ram Y

1,944
17
23

Ming Xu - MSFT · Answer 5 · 2016-07-01T05:02:07.887

5

If you mean you want to delete a container. I would like to suggest you to check http://msdn.microsoft.com/en-us/library/windowsazure/dd179408.aspx to see if Delete Container operation (The container and any blobs contained within it are later deleted during garbage collection) could fulfill the requirement.

edited Jul 01 '16 at 05:02

answered May 03 '12 at 07:54

Ming Xu - MSFT

2,116
1
11
13

The problem with this approach is that the Webrole would crash if the container is not found before create a new one with the same name. – Néstor Sánchez A. May 03 '12 at 18:30
In that case your only other option is to list all of the items in the container and delete them one at a time. – knightpfhor May 03 '12 at 22:10
@knightpfhor that's what I'm doing with Azure Storage Explorer (from Neudesic). But, despite I can select a group to delete together, it is veeeery slow. – Néstor Sánchez A. May 04 '12 at 01:10
Our web role will not crash unless we leave an exception unhandled. When doing anything with storage, please put it in a try catch block. Then, if something goes wrong, we can elegantly recover our application. In this particular scenario, if the container doesn’t exist, you can catch the exception and go on as normal. This will not affect your application logic since your goal is to make the container not exist. – Ming Xu - MSFT May 04 '12 at 09:38
Link-only answers are considered bad answers. Please flesh this out or delete. – spender Jun 30 '16 at 11:04

score 4 · Answer 6 · answered Jul 04 '17 at 13:50

If you are interested in a CLI way, then the following piece of code will help you out:

for i in `az storage blob list -c "Container-name" --account-name "Storage-account-name" --account-key "Storage-account-access-key" --output table | awk {'print $1'} | sed '1,2d' | sed '/^$/d'`; do az storage blob delete --name $i -c "Container-name" --account-name "Storage-account-name" --account-key "Storage-account-access-key" --output table; done

It first fetches the list of blobs in the container and deletes them one by one.

score 3 · Answer 7 · answered Aug 21 '17 at 17:51

If you are using a spark (HDInsight) cluster which has access to that storage account, then you can use HDFS commands on the command line;

hdfs dfs -rm -r wasbs://container_name@account_name.blob.core.windows.net/path_goes_here

The real benefit is that the cluster is unlikely to go down, and if you have screen running on it, then you won't lose your session whilst you delete away.

score 1 · Answer 8 · answered May 08 '12 at 12:40

1

For This case the better option is to identify the list of item found in the container. then delete each item from the container. That is the best option. If you delete the container you should have a run time error on the next time...

answered May 08 '12 at 12:40

Hope

1,252
4
17
35

score 1 · Answer 9 · answered Apr 22 '13 at 23:10

1

You can use Cloud Combine to delete all the blobs in your Azure container.

answered Apr 22 '13 at 23:10

ezolotko

1,723
1
21
21

How to clean an Azure storage Blob container?

9 Answers9

Linked