Delete files in batch from azure blob storage using service account

Question

I am using azure blob storage to store my project files.

I have a service account of azure blob storage(client_id and client_secret).I have created CloudBlobClient using StorageCredentialsToken as below:

StorageCredentialsToken credentialsToken = new StorageCredentialsToken("account name", "access token generated uing client_id and client_secret");
CloudBlobClient blobClient = new CloudBlobClient(new URI("https://accountname.blob.core.windows.net/"), credentialsToken);
CloudBlobContainer cloudBlobContainer = blobClient.getContainerReference("conteiner name");

Now using CloudBlobContainer I can delete one file at a time:

CloudBlockBlob blockBlobReference = cloudBlobContainer.getBlockBlobReference(key);
if (blockBlobReference.exists()) {
    blockBlobReference.delete();
}

How can I delete multiple files using a single call?

I find this doc which says we can delete multiple files with BlobBatchClient. In the document, I can not find any ways to create BlobBatchClient using a service account(using access token obtained by client_id and client_secret).

Can we delete files in async call as I need to delete 100s of files? Any alternative solutions to delete files in batch?

SDK version compile group: 'com.microsoft.azure', name: 'azure-storage', version: '8.6.5'

The [doc](https://learn.microsoft.com/en-us/java/api/overview/azure/storage-blob-batch-readme?view=azure-java-stable#creating-blobbatchclient) shows that you could create a BlobBatchClient from a BlobServiceClient. Have you tried to use [BlobServiceClient](https://learn.microsoft.com/en-us/python/api/azure-storage-blob/azure.storage.blob.blobserviceclient?view=azure-python#examples)? — unknown, Oct 19 '20 at 08:37
There is no way to create `BlobServiceClient` using the service account (`StorageCredentialsToken`) ref: https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/storage/azure-storage-blob#create-a-storage-account — Nitin, Oct 19 '20 at 08:40
Sorry, the link in my previous comment is about Python. I can just find the [method](https://github.com/Azure/azure-sdk-for-java/blob/master/sdk/storage/azure-storage-blob/src/samples/java/com/azure/storage/blob/SetMetadataAndHTTPHeadersExample.java#L35) using `StorageSharedKeyCredential` with account_name and account_key to create BlobServiceClient. — unknown, Oct 19 '20 at 09:12
@NitinVavdiya In the new java storage SDK v12, `StorageCredentialsToken` has been removed. If you want to use Azure AD auth to access blob, we need to provide a `TokenCredential ` object. For more details, please refer to https://github.com/Azure/azure-sdk-for-java/issues/6509 — Jim Xu, Oct 19 '20 at 09:18
Hey, to create `StorageSharedKeyCredential` we need accountKey. what is accountKey here? I tried with passing access token generated using client_id and client_secret but it is giving error `java.lang.IllegalArgumentException: Illegal base64 character 2e` It can not decode the access token — Nitin, Oct 19 '20 at 10:46

score 0 · Accepted Answer · answered Oct 19 '20 at 13:38

As per comment by Jim, I have created BlobServiceAsyncClient using access token sample method:

public void delete(List<String> files) {
        String endpoint = "https://azureaccount.blob.core.windows.net/";
        AccessToken accessToken = new AccessToken("access token created with client id and client secret", OffsetDateTime.now().plusHours(1)); 
        BlobServiceAsyncClient storageClient = new BlobServiceClientBuilder().credential(request -> Mono.just(accessToken))
                .endpoint(endpoint)
                .buildAsyncClient();
        BlobBatchClient blobBatchClient = new BlobBatchClientBuilder(storageClient).buildClient();
        List<String> blobUrls = new ArrayList<>();
        files.forEach(name -> {
            try {
                String blobUrl = endpoint + "conteinerName/" + URLEncoder.encode(name, "UTF-8");
                blobUrls.add(blobUrl);
            } catch (UnsupportedEncodingException e) {
                LOGGER.debug("Can not encode blob name={}", name);
            }
        });
        blobBatchClient.deleteBlobs(blobUrls, DeleteSnapshotsOptionType.INCLUDE).forEach(response -> {
                    LOGGER.debug("File with name={} deleted, status code={}", response.getRequest().getUrl(), response.getStatusCode());
                }
        );
}

Gradle dependencies:

compile group: 'com.azure', name: 'azure-storage-blob', version: '12.0.0'
compile group: 'com.azure', name: 'azure-storage-blob-batch', version: '12.6.0'

Each batch request supports a maximum of 256 subrequests. ref: https://learn.microsoft.com/en-us/rest/api/storageservices/blob-batch — Nitin, Oct 21 '20 at 09:39

Delete files in batch from azure blob storage using service account

1 Answers1