"Out of memory while expanding memory stream" error when attempting to download a file using TIdHTTP

Question

I am attempting to download a file from a web server using the standard TIdHTTP and TIdSSLIOHandler components in Delphi 10.4:

memorystream := TMemoryStream.Create;
http.request.useragent:='Mozilla/5.0 (Windows NT 5.1; rv:2.0b8) Gecko/20100101 Firefox/4.0b8';
http.HandleRedirects := True;
try
  http.get('https://dl.nmkd.de/ai/clip/coco/coco.ckpt',memorystream);
except
  on E:Exception do memo.lines.add(E.Message);
  memo.lines.add(http.responsetext);
end;
application.ProcessMessages;
memorystream.SaveToFile('coco.ckpt');
memorystream.free;

The responses I get in the memo are:

Out of memory while expanding memory stream
HTTP/1.1 200 OK

The file is around 9 GB in size.

How do I get a MemoryStream to be able to handle that size? I have more than enough physical RAM installed in the PC.

How about actually looking at the exception that is thrown? It could be something along "not enough memory" for 9 GiB... — AmigoJack, Jun 13 '21 at 18:27
Yes, that does help. Out of memory while expanding memory stream. I will update the question. — Some1Else, Jun 13 '21 at 18:34
You may have enough RAM installed, but that doesn't mean your app can use all of it. The file is 9GB in size. That means the `TMemoryStream` has to allocate *at least* 9GB of *consecutive* memory - and that is even assuming the web server provides the file size up front, otherwise `TMemoryStream` will have to allocate **much more** memory while growing dynamically while the download is in progress. Are you compiling for 32bit or 64bit? A 32bit app will never be able to allocate anywhere near this much memory. — Remy Lebeau, Jun 13 '21 at 19:17

score 4 · Accepted Answer · answered Jun 13 '21 at 19:01

Instead of loading the resource to memory temporarily, directly load it to the local file using a TFileStream:

Stream := TFileStream.Create('coco.cpkt');
try
  try
    http.get('https://dl.nmkd.de/ai/clip/coco/coco.ckpt', Stream);
  except
    on E:Exception do memo.lines.add(E.Message);
  end;
finally
  Stream.Free;
end;

"Out of memory while expanding memory stream" error when attempting to download a file using TIdHTTP

1 Answers1