Delphi: Copy FileStream to MemoryStream

Question

I want to copy a part of a FileStream to a MemoryStream.

FileStream.Write(Pointer(MemoryStream)^, MemoryStream.Size);
FileStream.Read(Pointer(MemoryStream)^, count);

Is that right? It isn't working for me.

to format your code, select it in the editor and press Control-K. — John Saunders, Jun 05 '10 at 00:51

score 19 · Accepted Answer · answered Jun 05 '10 at 00:18

19

You have to Read() from the FileStream into a separate buffer and then Write() that to the MemoryStream, ie:

var
  Buffer: PByte;

GetMem(Buffer, NumberOfBytes);
try
  FileStream.ReadBuffer(Buffer^, NumberOfBytes);
  MemoryStream.WriteBuffer(Buffer^, NumberOfBytes);
finally
  FreeMem(Buffer);
end;

Since you are dealing with two TStream objects, it would be easier to use the TStream.CopyFrom() method instead, ie:

MemoryStream.CopyFrom(FileStream, NumberOfBytes);

answered Jun 05 '10 at 00:18

Remy Lebeau

555,201
31
458
770

2

Thanks a lot! I used the CopyFrom, but I think your solutin will give me a better performance. Thanks again. – p.magalhaes Jun 05 '10 at 14:42
the TStream.CopyFrom() method uses a similar read-into-buffer-than-write-it approach internally, but does so with more error handling and buffer management than what I showed. – Remy Lebeau Jun 07 '10 at 23:47
ReadBuffer should be used when the number of bytes to read is known and fixed, better to use Read - it can actually return less bytes than the buffer size when there's no more. I'd write: BytesRead := FileStream.Read(Buffer^, NumberOfBytes); MemoryStream.Write(Buffer^, BytesRead); – Jun 16 '10 at 07:49
Since the OP is wanting to copy a portion of a file, presumably the OP does know ahead of time how many bytes to copy. – Remy Lebeau Jun 16 '10 at 22:53
2

Just wanted to help anyone who might encounter the same problem as I did. Using solution Remy posted is MUCH, MUCH faster for very large files (> 1.5GB) than using MemoryStream.CopyFrom method. I spent 2 days figuring it out... :( – Nikola Malešević Apr 07 '11 at 15:24
1

Better stick to Write, CopyFrom seems about 5 times slower. – hikari Dec 13 '12 at 07:46

score 2 · Answer 2 · answered Mar 24 '18 at 21:14

The following solution does not use a separate buffer as the solution that was already posted. Instead it writes directly to the buffer of the destination memory stream. This is faster because the other solution copies twice, first into the temporary buffer and finally into the memory stream.

...
try
  MemoryStream.SetSize(NumberOfBytes); // Allocating buffer
  FileStream.ReadBuffer(MemoryStream.Memory^, NumberOfBytes);
finally
  MemoryStream.Free();
...

This works because SetSize also allocates the buffer of the memory stream. See SetSize documentation.

Use SetSize to set the Size of a memory stream before filling it with data. SetSize allocates the memory buffer to hold NewSize bytes [...].

I also tested the solution with CopyFrom, but that solution is very slow working with giant files because it seems to use a very small buffer.

If files are to great to read directly with the method above it can be done with an own function that reads chunks directly to the memory stream. In order to be faster than the CopyFrom method, these chunks should be bigger. The following code uses a flexible buffer e.g. 256 MiB. Please feel free to make a function out of it.

var
  ...
  MemoryStreamPointer: Pointer;
  BlockSize: Integer;
  BytesToRead: Integer;
  BytesRead: Integer;
  RemainingBytes: Integer;

begin
  ...
  BlockSize := 256 * 1024 * 1024; // 256 MiB block size

  MemoryStream.SetSize(NumberOfBytes); // Allocating buffer
  MemoryStreamPointer := MemoryStream.Memory;

  RemainingBytes := NumberOfBytes;
  while RemainingBytes > 0 do
  begin
    BytesToRead := min(RemainingBytes, BlockSize);
    BytesRead := FileStream.Read(MemoryStreamPointer^, BytesToRead);
    RemainingBytes := RemainingBytes - BytesRead;
    MemoryStreamPointer := Pointer(NativeInt(MemoryStreamPointer) + BytesRead);
  end;
  ...
end;

Please take caution that the above code contains no error handling. Further think about setting the file streams position to 0 before reading.

Delphi: Copy FileStream to MemoryStream

2 Answers2