Abandoned multi-part uploads to Azure Blob Storage

Question

We are uploading data to Azure Blob Storage using Microsoft's Java client library. First we create a blob for uploading:

CloudBlobContainer container = client.getContainerReference(containerName);
CloudBlockBlob blob = container.getBlockBlobReference(nativePath);

Each part to be uploaded is done with code like this:

try (InputStream inputStream = new FileInputStream(part.path.toFile())) {
  String blockId = String.format("%05d", part.index);
  String blockIdEncoded = DatatypeConverter.printBase64Binary(blockId.getBytes());
  blob.uploadBlock(blockIdEncoded, inputStream, part.size);
}

at the end, all of the parts are stitched together into a blob using:

blob.commitBlockList(blockList);

What happens if this process is abandoned (due to termination or logic error) before the commitBlockList() is called? Do the parts leak? Are they garbage collected eventually? Can I see them somewhere on the Azure portal?

How should I terminate this process gracefully so that all of the uploaded-but-unused parts are deleted?

Answer 1

What happens if this process is abandoned (due to termination or logic error) before the commitBlockList() is called?

Any blocks you upload and not committed remain in Azure Storage for a period of 7 days.

Are they garbage collected eventually?

Yes, after 7 days.

Can I see them somewhere on the Azure portal?

No. You cannot see these blocks in Azure portal. However you can get the list of blocks for blob by calling Get Block List REST API operation (or equivalent SDK operation).

How should I terminate this process gracefully so that all of the uploaded parts are deleted?

Currently there's no way for you to delete the uncommitted blocks. One way to solve this problem is to upload a zero byte blob with the same name and delete that blob. When you upload a blob with the same name, the existing blocks will be deleted.

You can read more about it here: https://learn.microsoft.com/en-us/rest/api/storageservices/put-block#remarks.