Streaming HttpResponse through Owin

Question

I have a HttpResponse object as a result of HttpClient.SendAsync() call. The response has a chunked transfer encoding and results in 1.5 GB of data.

I want to pass this data through OWIN pipeline. To do this I need to convert it to a stream. Simplified code to do this is:

public async Task Invoke(IDictionary<string, object> environment)
{
    var httpContent = GetHttpContent();
    var responseStream = (Stream)environment["owin.ResponseBody"];
    await httpContent.CopyToAsync(responseStream);
}

However, the last line results in copying the entire stream to the memory. And when I use wget to download the data directly from the backend server, it is downloaded successfully and shows a progress bar (although it doesn't know the overall size since it is chunked). But when I use wget to download data from my OWIN-hosted application it sticks on sending the request.

How should I stream this data through an OWIN pipeline to prevent copying it to memory?

EDIT

This is how I get the HttpResponse:

var client = new HttpClient(new HttpClientHandler());
// …and then:
using (var request = new HttpRequestMessage { RequestUri = uri, Method = HttpMethod.Get })
{
    return client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).Result;
}

Can you show your HttpClient call? Most HttpClient methods buffer the response by default. You need one that doesn't, like GetStreamAsync or SendAsync with HttpCompletionOption.ResponseHeadersRead. — Tratcher, Dec 10 '16 at 00:07

score 0 · Answer 1 · answered Dec 14 '16 at 14:44

0

I assume this is in IIS? System.Web also buffers responses: https://msdn.microsoft.com/en-us/library/system.web.httpresponse.bufferoutput(v=vs.110).aspx

See server.DisableResponseBuffering in https://katanaproject.codeplex.com/wikipage?title=OWIN%20Keys&referringTitle=Documentation

answered Dec 14 '16 at 14:44

Tratcher

5,929
34
44

Yes, it is hosted on IIS. But we don't use System.Web at all (we even don't have reference to it). – bpiec Dec 16 '16 at 10:01
OWIN on IIS uses System.Web (Microsoft.Owin.Host.SystemWeb) – Tratcher Dec 16 '16 at 14:46

Streaming HttpResponse through Owin

1 Answers1