4

I'm trying to use an Azure function to return a file from blob storage. As it stands, I've got it working, but it works inefficiently by reading the entire blob into memory, then writing it back out. This works for small files, but once they get big enough, it's very inefficient.

How can I make my function return the blob directly without it having to be read entirely into memory?

Here's what I'm using currently:

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, Binder binder, TraceWriter log)
{
    // parse query parameter
    string fileName = req.GetQueryNameValuePairs()
        .FirstOrDefault(q => string.Compare(q.Key, "name", true) == 0)
        .Value;

    string binaryName = $"builds/{fileName}";

    log.Info($"Fetching {binaryName}");

    var attributes = new Attribute[]
    {    
        new BlobAttribute(binaryName, FileAccess.Read),
        new StorageAccountAttribute("vendorbuilds")
    };

    using (var blobStream = await binder.BindAsync<Stream>(attributes))
    {
        if (blobStream == null) 
        {
            return req.CreateResponse(HttpStatusCode.NotFound);
        }

        using(var memoryStream = new MemoryStream())
        {
            blobStream.CopyTo(memoryStream);
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Content = new ByteArrayContent(memoryStream.ToArray());
            response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") { FileName = fileName };
            response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
            return response;
        }
    }
}

My function.json file:

{
  "bindings": [
    {
      "authLevel": "anonymous",
      "name": "req",
      "type": "httpTrigger",
      "direction": "in",
      "methods": [
        "get",
        "post"
      ]
    },
    {
      "name": "$return",
      "type": "http",
      "direction": "out"
    }
  ],
  "disabled": false
}

I'm not a C# developer, nor an Azure developer, so most of this stuff escapes me.

Anass Kartit
  • 2,017
  • 15
  • 22
Dale Myers
  • 2,703
  • 3
  • 26
  • 48
  • 3
    Can't you just return a link to the file along with a shared access signature? – CSharpRocks Apr 03 '19 at 16:07
  • @CSharpRocks Honestly, I didn't even know this was a thing... This is probably what I need. If you write this as an answer, I'm happy to accept it. Thanks! – Dale Myers Apr 03 '19 at 20:08

3 Answers3

6

I did like below for minimum memory footprint (without keeping the full blob into bytes in memory). Note that instead of binding to stream, I am binding to a ICloudBlob instance (luckily, C# function supports several flavors of blob input binding) and returning open stream. Tested it using memory profiler and works fine with no memory leak even for large blobs.

NOTE: You don't need to seek to stream position 0 or flush or dispose (disposing would be automatically done on response end);

using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Azure.Storage.Blob;

namespace TestFunction1
{
   public static class MyFunction
   {
        [FunctionName("MyFunction")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = "video/{fileName}")] HttpRequest req,
            [Blob("test/{fileName}", FileAccess.Read, Connection = "BlobConnection")] ICloudBlob blob,
            ILogger log)
        {
            var blobStream = await blob.OpenReadAsync().ConfigureAwait(false);
            return new FileStreamResult(blobStream, "application/octet-stream");
        }
   }
}
krishg
  • 5,935
  • 2
  • 12
  • 19
  • Why are you using ConfigureAwait(false), does it cause any problems if you don't use this? – adR Aug 12 '21 at 10:47
  • @adR , https://devblogs.microsoft.com/dotnet/configureawait-faq/ – krishg Aug 13 '21 at 11:55
  • You can better link to the correct paragraph: https://devblogs.microsoft.com/dotnet/configureawait-faq/#what-does-configureawaitfalse-do This documentation then says you need a good reason to do so, which OP has not really provided. – Rich_Rich Dec 22 '22 at 14:34
2

I like @CSharpRocks suggestion of creating a SAS and returning the link to blob storage, but I also found this article that might be relevant:

https://anthonychu.ca/post/azure-functions-static-file-server/

Here is the relevant code:

var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(memoryStream);
response.Content.Headers.ContentType = 
    new MediaTypeHeaderValue("application/octet-stream");
return response;
typheon
  • 138
  • 1
  • 1
  • 9
  • I'm pretty sure this is what I tried originally, but discovered that it didn't work: https://github.com/Azure/azure-functions-host/issues/1666 – Dale Myers Apr 03 '19 at 20:04
0

Thanks @krishg for your answer!

Based on your code I figured out how to do the opposite, posting binary data from in my case Unity to a blob file as clean as possible.

[FunctionName("Upload")]
        public static async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "options", "put", Route = "upload/{name}")] HttpRequest req,
        [Blob("container/{name}.bin", FileAccess.Write)] ICloudBlob blob, ILogger log)
        {
            if (req.Method == "OPTIONS")
            {                
                req.HttpContext.Response.Headers.Add("access-control-allow-methods", "PUT, OPTIONS");
                req.HttpContext.Response.Headers.Add("access-control-allow-headers", "Content-Type");
                return new EmptyResult();
            }

            await blob.UploadFromStreamAsync(req.Body).ConfigureAwait(false);            
            return new OkObjectResult("data saved");
        }
bartburkhardt
  • 7,498
  • 1
  • 18
  • 14