Can an Azure Function timer trigger iterate all blobs in a container or directory?

Question

Is it possible to iterate through and operate on all of the blobs in an Azure storage container using a function triggered on a regular interval (timer trigger).

Path: {container-name}/{directory-name}/{file-name}

using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;

namespace SampleNamespace
{
    public static class SampleFunction
    {
        [FunctionName("SampleFunction")]
        public static void Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
        {
            // How to iterate an operate on all blobs?
        }
    }
}

You can retrieve all containers and then iterate through them.... this may help you https://stackoverflow.com/questions/8458720/how-to-list-the-containers-in-azure-blob-storage — Fabrizio Migotto, Mar 05 '20 at 17:45
Does this answer your question? [Iterating all blobs in a container using Azure Function](https://stackoverflow.com/questions/44056687/iterating-all-blobs-in-a-container-using-azure-function) — user1672994, Mar 05 '20 at 17:46
@user1672994 that seems outdated. I tried that answer but it looks like there has been a change around ListBlobsSegmentedAsync(). A working example using the API for today would be very helpful. — crayden, Mar 05 '20 at 17:53

score 1 · Accepted Answer · answered Mar 06 '20 at 01:23

You should install this blob storage nuget package Microsoft.Azure.Storage.Blob for the azure function. Then you can use the sync method like ListBlobs instead of async method ListBlobsSegmentedAsync. At last, you can write the code to operate these blobs.

The code like below:

using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using System;

namespace FunctionApp7
{
    public static class Function1
    {
        [FunctionName("Function1")]
        public static void Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
        {
            log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");

            log.LogInformation("the blobs list:");

            var connectionString = "DefaultEndpointsProtocol=https;AccountName=xx;AccountKey=xxxx;EndpointSuffix=core.windows.net";
            var containerName = "test4";
            var directoryName = "sub1";

            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);

            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

            CloudBlobContainer container = blobClient.GetContainerReference(containerName);

            //list blobs in a container
            var blobs = container.ListBlobs(useFlatBlobListing: true);

            foreach (IListBlobItem item in blobs)
            {
                if (item.GetType() == typeof(CloudBlockBlob))
                {
                    CloudBlockBlob blob = (CloudBlockBlob)item;
                    //other operation.
                }
            }

            //list blobs in a directory
            CloudBlobDirectory directory = container.GetDirectoryReference(directoryName);
            var blobs_2 = directory.ListBlobs(useFlatBlobListing: true);

            foreach(IListBlobItem item in blobs_2)
            {
                if (item.GetType() == typeof(CloudBlockBlob))
                {
                    CloudBlockBlob blob = (CloudBlockBlob)item;
                    //other operation.
                }
            }


        }
    }
}

score 0 · Answer 2 · answered Mar 06 '20 at 09:06

Except the way with the storage blob sdk(Ivan provide), you could use the blob binding to get the container client and then do the list opertaion.

You could refer to the blob input usage, you could find it supports bind the CloudBlobContainer or CloudBlobDirectory type. You could refer to below code.

[FunctionName("Function1")]
public static async System.Threading.Tasks.Task RunAsync([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log,
            [Blob("test",Connection = "AzureWebJobsStorage")]CloudBlobContainer container)
        {
            log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
            var segment = await container.ListBlobsSegmentedAsync(null);
            var blobs = segment.Results;
            foreach (var blob in blobs) {

                log.LogInformation(blob.GetType().ToString(), blob.ToString());
            }
        }

If will return all blobs and blob directory, the below is my test result, so you need to do a judge if return type is CloudBlockBlob or CloudBlobDirectory then do the list operation.

Can an Azure Function timer trigger iterate all blobs in a container or directory?

2 Answers2