2

We have a webservice that serves up files. Recently, we have come across a Very Large File - more than 2 GB - that can't be copied into the buffer. I've modified the code to use HttpCompletionOptions.ResponseHeadersRead to not use the buffer and copy directly to a stream. However, most of the time I get

System.IO.IOException: 'Unable to read data from the transport connection: The connection was closed.'

Curl is able to download it without problem. The exception doesn't happen every time, but it's most of the time. I set HttpClient.Timeout to an hour, so that's not the problem. The Exception itself is very ambiguous and I can't find any reason that it would be closing the connection. The logs on the web server also say

info: Microsoft.AspNetCore.Server.Kestrel[34] Connection id "0HLO4L4D3UAMS", Request id "0HLO4L4D3UAMS:00000001": the application aborted the connection.

so it seems to be something on the client side.

var requestMessage = GetVeryLargeFile(asset, HttpMethod.Get);
using (var result = await _client.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false))
{
   result.EnsureSuccessStatusCode();
   cancellationToken.ThrowIfCancellationRequested();
   using (var stream = await result.Content.ReadAsStreamAsync().ConfigureAwait(false))
   {
      cancellationToken.ThrowIfCancellationRequested();
      using (var fileStream = _fileProvider.Create(filePath))
      {
         await stream.CopyToAsync(fileStream.StreamInstance).ConfigureAwait(false);
         if (fileStream.Length == 0)
         {
            throw new DataException($"No data retrieved for {asset.Url}");
         }
      }
   }
}

UPDATE: Based on comments here, I changed the copy line to be synchronous, and that fixed the error. That's certainly less than optimal, but I'm still struggling to figure out why the async will randomly close the connection.

stream.CopyTo(fileStream.StreamInstance);
Kyle W
  • 3,702
  • 20
  • 32
  • Could be that the file is very large and the browser times out when downloading. Any reason not to just redirect to the file and then they can download? – Richard Housham Jul 12 '19 at 14:14
  • @RichardHousham It's not a user, it's a service. There's no browser involved. – Kyle W Jul 12 '19 at 14:19
  • yeah, but your still serving a file. Right? So instead of the filestream etc. Just a redirect to the file, perhaps copy to a temp directory. – Richard Housham Jul 12 '19 at 14:22
  • @RichardHousham No, I'm not serving anything. This code copies an image to a hard drive that is shipped – Kyle W Jul 12 '19 at 14:25

0 Answers0