ASP.NET Core 3.1 read stream file upload in HttpPut request problem

Question

Problem Statement:

I'm trying to iterate over a Streamed file upload in a HttpPut request using the Request.Body stream and I'm having a real hard time and my google-fu has turned up little. The situation is that I expect something like this to work and it doesn't:

[HttpPut("{accountName}/{subAccount}/{revisionId}/{randomNumber}")]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status500InternalServerError)]
public async Task<IActionResult> PutTest()
{
    var memStream = new MemoryStream();
    var b = new Memory<byte>();
    int totalBytes = 0;
    int bytesRead = 0;
    byte[] buffer = new byte[1024];

    do
    {
        bytesRead = await Request.Body.ReadAsync(new Memory<byte>(buffer), CancellationToken.None);
        totalBytes += bytesRead;
        await memStream.WriteAsync(buffer, 0, bytesRead);
    } while (bytesRead > 0);
    
    return Ok(memStream);
}

In the debugger, I can examine the Request.Body and look at it's internal _buffer. It contains the desired data. When the above code runs, the MemoryStream is full of zeros. During "Read", the buffer is also full of zeros. The Request.Body also has a length of 0.

The Goal:

Use a HttpPut request to upload a file via streaming, iterate over it in chunks, do some processing, and stream those chunks using gRPC to another endpoint. I want to avoid reading the entire file into memory.

What I've tried:

This works:

using (var sr = new StreamReader(Request.Body))
{
    var body = await sr.ReadToEndAsync();
    return Ok(body);
}

That code will read all of the Stream into memory as a string which is quite undesirable, but it proves to me that the Request.Body data can be read in some fashion in the method I'm working on.

In the configure method of the Startup.cs class, I have included the following to ensure that buffering is enabled:

app.Use(async (context, next) => {
                context.Request.EnableBuffering();
                await next();
            });

I have tried encapsulating the Request.Body in another stream like BufferedStream and FileBufferingReadStream and those don't make a difference.

I've tried:

var reader = new BinaryReader(Request.Body, Encoding.Default);
do
{
    bytesRead = reader.Read(buffer, 0, buffer.Length);
    await memStream.WriteAsync(buffer);
} while (bytesRead > 0);

This, as well, turns up a MemoryStream with all zeros.

Streams won't report a correct length if `.CanSeek == false`. You shouldn't need to create a `new Memory` on every iteration, but you might want to `.Slice` it for processing / writing. Returning `Ok(MemoryStream)` doesn't make a lot of sense to me, but I assume that's irrelevant. — Jeremy Lakeman, May 26 '21 at 03:33
Thanks for the feedback. Yes, the `Ok(MemoryStream)` is irrelevant. I can confirm `.CanSeek` is `true`. I'll try the slice idea though. — Tim Frye, May 26 '21 at 03:49
When sending the message you need to add to the request the content length which is totalBytes. — jdweng, May 26 '21 at 05:56

Rodrigo Rodrigues · Answer 1 · 2021-05-26T06:36:33.630

I use to do this kind of request body stream a lot in my current project.

This works perfectly fine for me:

[HttpPut("{accountName}/{subAccount}/{revisionId}/{randomNumber}")]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status500InternalServerError)]
public async Task<IActionResult> PutTest(CancellationToken cancel) {
  using (var to = new MemoryStream()) {
    var from = HttpContext.Request.Body;
    var buffer = new byte[8 * 1024];
    long totalBytes = 0;
    int bytesRead;
    while ((bytesRead = await from.ReadAsync(buffer, 0, buffer.Length, cancel)) > 0) {
      await to.WriteAsync(buffer, 0, bytesRead, cancel);
      totalBytes += bytesRead;
    }
    return Ok(to);
  }
}

The only things I am doing different are:

I am creating the MemoryStream in a scoped context (using).
I am using a slightly bigger buffer (some trial and error led me to this specific size)
I am using a different overload of Stream.ReadAsync, where I pass the bytes[] buffer, the reading length and the reading start position as 0.

ASP.NET Core 3.1 read stream file upload in HttpPut request problem

Problem Statement:

The Goal:

What I've tried:

1 Answers1