Stream multipart request body without waiting for it to upload first

Question

Is it possible to send a POST request to a controller and read the HttpContext.Request.Body stream immediately after sending the POST request (for example if the file is 10GB large)? If I have a form like this:

<form method="post" enctype="multipart/form-data">

and a controller like this:

public class FileController : ControllerBase 
{
    [HttpPost("[action]")]
    public async Task Upload() 
    {
        var stream = HttpContext.Request.Body;
    }
}

If I submit a file, it uploads the file first (as I can see in my browser), and then it hits the controller afterwards, with the whole file saved in a temporary file, as you can see here: https://github.com/aspnet/HttpAbstractions/blob/07d115400e4f8c7a66ba239f230805f03a14ee3d/src/Microsoft.AspNetCore.WebUtilities/FileBufferingReadStream.cs#L186

I can verify that it saves the uploaded file to disk, which won't work for my application, as I don't have unlimited storage. My end goal is to upload the file to FTP, so the request goes like this:

client -> service -> FTP

What am I doing wrong?

Check out https://stackoverflow.com/questions/36437282/dealing-with-large-file-uploads-on-asp-net-core-1-0. Seems like you could read MultipartSection stream and direct the content toward FTP service. — Kiryl, Jul 10 '20 at 10:09
@KirylZ I actually already use MultipartReader and so on, but the problem is that the code isn't being hit in my controller before after upload is done. — MortenMoulder, Jul 10 '20 at 12:15
You mean before temp file is created on server side? Have u tried to remove default binders? — Kiryl, Jul 10 '20 at 14:00
Like in here https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.1#upload-large-files-with-streaming — Kiryl, Jul 10 '20 at 14:06
@KirylZ Sorry for the late reply. That actually worked. If you want to answer so I can select it as the answer, go ahead! — MortenMoulder, Jul 28 '20 at 11:51

score 1 · Accepted Answer · answered Jul 28 '20 at 15:25

1

A range of solutions is described at Dealing with large file uploads on ASP.NET Core 1.0.

There is one important thing though that is not mentioned there - you have to remove the default binders. Otherwise buffering will kick in. Here is is how - Upload large files with streaming.

answered Jul 28 '20 at 15:25

Kiryl

1,416
9
21

1

Thanks again - it worked perfectly. I need to do a bit of hacking to get it to work, because of this line `Since model binding doesn't read the form, parameters that are bound from the form don't bind (query, route, and header continue to work)`. Meaning I cannot pass in any POST parameters or form data. I've added my necessary stuff in the header instead, which works flawlessly, but is kind of a hack. – MortenMoulder Jul 29 '20 at 08:04
So u're trying to pass several parameters in a request? Like the file and something else? – Kiryl Jul 29 '20 at 08:17
Basically, yeah. File is one of them and the path where to save the file is the other one. I had to add an `X-Upload-Path` header along with the request to pass that with it. The second I start doing anything like `HttpContext.Request.Form` it will load the entire uploaded file into memory or on disk – MortenMoulder Jul 29 '20 at 09:44
1

I would suggest to pass all the parameters in a body and divide them with boundary and Content-Disposition header. Then you'll get them separately as Sections by calling MultipartReader.ReadNextSectionAsync(). – Kiryl Jul 29 '20 at 11:50
Yeah that would definitely be a good option as well. However, I only need the path so I can't be too bothered by it – MortenMoulder Jul 29 '20 at 11:59

Stream multipart request body without waiting for it to upload first

1 Answers1