5

Does anyone know of a composite stream solution that will pre-load the first portion of a Stream in to a MemoryStream and keep the remainder as the original Stream which will be accessed when subsequent parts are required as necessary?

I should imagine some wrapper class would implement the Stream interface and transparently juggle the access between the two streams depending upon which part is accessed.

I'm hoping this is a solution someone may have solved before, maybe to optimize performance of reading a large FileStream.

In my case I'm trying to get around a Windows Phone 8 bug reading large files from the SD card. More detail of the issue I'm trying to circumnavigate is provided in this answer: https://stackoverflow.com/a/17355068/250254

Community
  • 1
  • 1
Gavin
  • 5,629
  • 7
  • 44
  • 86
  • Take a look at http://stackoverflow.com/questions/3879152/how-do-i-concatenate-two-system-io-stream-instances-into-one – Russ Cam Jan 29 '14 at 15:00
  • @RussCam - thanks for that, much appreciated! Not exactly the same problem as my streams will overlap but I'll have a pick through and see if there's something I can adapt. – Gavin Feb 03 '14 at 19:44
  • 1
    I don't think this is the right way to solve the problem... how much data do you intend to load into the MemoryStream anyway? If I understood the issue correctly, the Seek problem occurs for any offset in the lower 32 bits of a long, which means you would need to load 4GB of data into memory... – Thomas Levesque Feb 03 '14 at 20:30
  • @ThomasLevesque - Experimenting it seems that around the 400408 position the issue might go away. So about 0.4 mb would need to be kept in the memory stream. Still experimenting so I could be wrong! – Gavin Feb 03 '14 at 22:03

1 Answers1

5

There isn't any reasonable way you can use a MemoryStream to work around the bug, you'll fall over on OutOfMemoryException first. Let's focus a bit on the bug, I'll simplify the code a bit to make it readable:

DistanceToMove = (offset & 0xffffffff00000000L) >> 32;
DistanceToMoveHigh = offset & 0xffffffffL;
SetFilePointer(this.m_handle, lDistanceToMove, ref lDistanceToMoveHigh, begin);

The Microsoft programmer accidentally swapped the low and high values. Well, so can you to undo the bug. Swap them yourself so the bug swaps them back the way you want it:

public static void SeekBugWorkaround(Stream stream, long offset, SeekOrigin origin) {
    ulong uoffset = (ulong)offset;
    ulong fix = ((uoffset & 0xffffffffL) << 32) | ((uoffset & 0xffffffff00000000L) >> 32);
    stream.Seek((long)fix, origin);
}

In case it needs to be said, it apparently does, you do have to count on Microsoft eventually fixing this bug. Hard to predict when so gamble on the next point release. There are some odds you can auto-detect this, albeit that it isn't obvious what Microsoft is going to do since this bug is so breaking. The return value of Seek() as well as the Position property return value suffer from the same bug. So seek to position 1 and verify that you get 1 back.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
  • Yikes, this sounds like a forward-compatibility nightmare! – Cory Nelson Feb 03 '14 at 22:04
  • @CoryNelson - Should be able to check Windows Phone OS version I would hope so bypass the fix in future release. – Gavin Feb 03 '14 at 22:06
  • @hans-passant - Thanks for that - much appreciated. I'll have a play in the next day or two to see if I can get it to work (it's for a non-paying side project I do). I guess I'll implement a wrapper stream class that overrides the Seek method using your logic. – Gavin Feb 03 '14 at 22:10
  • 1
    I wonder how a bug like this managed to go to production undetected... Don't they have unit tests at MS? – Thomas Levesque Feb 03 '14 at 22:10
  • For anyone that would like a the stream wrapper that can be dropped in to their solution - https://github.com/gavinharriss/ExternalStorageFileWrapper-wp8 – Gavin Feb 07 '14 at 03:56