Unzip files and save them to a Blob Storage stream

Question

My workflow for this proof of concept is:

1. Azure Function App detects a .zip file in blob storage (Stream inputBlob)
2. Function app calls my code, which needs to extract the the files and save them individually to the Blob Storage Container Stream outputBlob

The code does get the .zip from inputBlob, and I can see in the debugger that the ZipArchive contains the contents of the .zip. However no files are output, with no errors. What do I need to do to save all of the files to the outputBlob stream? I'm sure I'm missing something related to the stream copying.

[FunctionName("Name")]
public static void Run(
    [BlobTrigger("input/{name}", Connection = "AzureWebJobsStorage")]Stream inputBlob,
    [Blob("output/{name}", FileAccess.Write)] Stream outputBlob,
    string name, ILogger log)
{
    try
    {
        using var zip = new ZipArchive(inputBlob);
        
        foreach (var item in zip.Entries)
        {
            using var stream = item.Open();
            stream.CopyTo(outputBlob);
            stream.Close();
        }

        outputBlob.Seek(0, SeekOrigin.Begin);
        outputBlob.Close();
    }
    catch (Exception ex)
    {
        log.Log(LogLevel.Error, $"Error at {name}: {ex.Message}");
        throw;
    }
}

Answer 1

We can debug from the VSCode to know where it is causing the issue, for that we need to add AzureWebJobsStorage to UseDevelopmentStorage=true in the local.settings.json file

Below is the local.settings.json file looks like:

{
    "IsEncrypted": false,
    "Values": {
        "AzureWebJobsStorage": "UseDevelopmentStorage=true",
        "FUNCTIONS_WORKER_RUNTIME": "dotnet",
        "unziptools_STORAGE": "DefaultEndpointsProtocol=https;AccountName=unziptools;AccountKey=XXXXXXXXX;EndpointSuffix=core.windows.net",
    }
}

Similar way as you defined, we need to specify blobtrigger :

[BlobTrigger("input-files/{name}", Connection = "cloud5mins_storage")]Stream myBlob

Also get the destination container to load the unzip files:

    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(destinationStorage);
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
    CloudBlobContainer container = blobClient.GetContainerReference(destinationContainer);

Below is the sample code for getting the input blob and placing it in the destination storage and container.

public static async Task Run([BlobTrigger("input-files/{name}", Connection = "cloud5mins_storage")]CloudBlockBlob myBlob, string name, ILogger log)
{
    log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name}");

    string destinationStorage = Environment.GetEnvironmentVariable("destinationStorage");
    string destinationContainer = Environment.GetEnvironmentVariable("destinationContainer");

    try{
        if(name.Split('.').Last().ToLower() == "zip"){

            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(destinationStorage);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference(destinationContainer);
            
            using(MemoryStream blobMemStream = new MemoryStream()){

                await myBlob.DownloadToStreamAsync(blobMemStream);

                using(ZipArchive archive = new ZipArchive(blobMemStream))
                {
                    foreach (ZipArchiveEntry entry in archive.Entries)
                    {
                        log.LogInformation($"Now processing {entry.FullName}");

                        //Replace all NO digits, letters, or "-" by a "-" Azure storage is specific on valid characters
                        string valideName = Regex.Replace(entry.Name,@"[^a-zA-Z0-9\-]","-").ToLower();

                        CloudBlockBlob blockBlob = container.GetBlockBlobReference(valideName);
                        using (var fileStream = entry.Open())
                        {
                            await blockBlob.UploadFromStreamAsync(fileStream);
                        }
                    }
                }
            }
        }
    }
    catch(Exception ex){
        log.LogInformation($"Error! Something went wrong: {ex.Message}");

    }            
}

We have a blog where we have detail information about this, thanks to frankynotes.

Answer 2

The accepted answer was mostly there for what I needed, but had some non-functional code and checks I didn't need. For documentation, I'm putting the working modified code here:

[FunctionName("name")]
public static async Task Run(
    [BlobTrigger("input-blob-container/{name}", Connection = "AzureWebJobsStorage")]Stream inputBlob,
    string name, ILogger log)
{
    try
    {
        log.Log(LogLevel.Information, "Starting unzip.");

        string destinationStorage = Environment.GetEnvironmentVariable("destinationStorage");
        string destinationContainer = Environment.GetEnvironmentVariable("destinationContainer");

        CloudStorageAccount storageAccount  = CloudStorageAccount.Parse(destinationStorage);
        CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
        CloudBlobContainer container = blobClient.GetContainerReference(destinationContainer);

        using ZipArchive archive = new ZipArchive(inputBlob);
        foreach (ZipArchiveEntry entry in archive.Entries)
        {
            log.LogInformation($"Now processing {entry.FullName}");
            
            CloudBlockBlob blockBlob = container.GetBlockBlobReference(entry.Name);
            await using var fileStream = entry.Open();
            await blockBlob.UploadFromStreamAsync(fileStream);
        }

        log.Log(LogLevel.Information, "Finished unzip.");
    }
    catch (Exception ex)
    {
        log.Log(LogLevel.Error, $"Error at {name}: {ex.Message}");
        throw;
    }
}

And my local.settings.json

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "destinationStorage": "DefaultEndpointsProtocol=https;AccountName=azuresitename;AccountKey=;BlobEndpoint=https://azuresitename.blob.core.windows.net/;TableEndpoint=https://azuresitename.table.core.windows.net/;QueueEndpoint=https://azuresitename.queue.core.windows.net/;FileEndpoint=https://azuresitename.file.core.windows.net/",
    "destinationContainer" : "output-blob-container" 
  }
}

I added the destinationStorage and destinationContainer settings via VS Code using the Azure extension (detailed in the video in the accepted answer's blog post).