3

I need to open read disk in raw mode, so I'm using CreateFile API function for that purposes.

private static FileStream OpenDisk(string drive)
{
    // Try to open hard disk drive in raw mode for reading
    SafeFileHandle safeHandle = Native.CreateFile(
        string.Format(@"\\.\{0}", drive),
        FileAccess.Read,
        FileShare.Read,
        IntPtr.Zero,
        FileMode.Open,
        FileAttributes.ReadOnly | FileAttributes.Device,
        IntPtr.Zero);

    // Check if the drive was successfully opened
    if (safeHandle.IsInvalid)
    {
        Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
    }

    // Create file stream on the file for reading
    return new FileStream(safeHandle, FileAccess.Read);
}

But when I try to read from the stream I get the following error

Handle does not support synchronous operations. The parameters to the FileStream constructor may need to be changed to indicate that the handle was opened asynchronously (that is, it was opened explicitly for overlapped I/O).

Here is the sample code that reproduces this issue

using (FileStream stream = OpenDisk("X:"))
{
    byte[] buffer = new byte[1000];
    while (stream.Read(buffer, 0, 1000) > 0) { }
}

I don't really get what does it want from me? It works when I use larger buffer (for example 4096). When I say it works I mean it really works (always), it was working for a while, until I changed buffer size. I guess it does some kind of asynchronous buffering inside and when I specify larger buffer then default buffering size it's just not used, but how to get rid of this?

Thanks

update

When I try to read it using BufferedStream I get the same issue

using (BufferedStream stream = new BufferedStream(OpenDisk("X:"), 4096))
{
    byte[] buffer = new byte[1000];
    while (stream.Read(buffer, 0, 1000) > 0) { }
}

Do I understand wrong purpose of BufferedStream? Shouldn't it read and cache chunks of specified size?

axe
  • 2,331
  • 4
  • 31
  • 53

1 Answers1

4

In short, reads from volumes are non-buffered and must be of whole sectors. A buffer size of 4096 achieves this. The error message is misleading because asynchronous I/O and non-buffering tend to go together. Some details:

From the CreateFile docs:

Volume handles can be opened as noncached at the discretion of the particular file system, even when the noncached option is not specified in CreateFile. You should assume that all Microsoft file systems open volume handles as noncached. The restrictions on noncached I/O for files also apply to volumes.

From the File Buffering docs:

  • File access sizes, including the optional file offset in the OVERLAPPED structure, if specified, must be for a number of bytes that is an integer multiple of the volume sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1,024, 1,536, or 2,048 bytes, but not of 335, 981, or 7,171 bytes.
  • File access buffer addresses for read and write operations should be physical sector-aligned, which means aligned on addresses in memory that are integer multiples of the volume's physical sector size. Depending on the disk, this requirement may not be enforced.
arx
  • 16,686
  • 2
  • 44
  • 61
  • Thanks. Now looks more clear, but there is still one question. Why it doesn't work if I use BufferedStream with 4096 specified as buffer size. See update in question. – axe Sep 26 '12 at 18:50
  • The BufferedStream has a buffer of 4096 bytes, but that doesn't imply it reads data in 4096 byte chunks. This might seem silly, but if you had a very large buffer (e.g. 10MB) you wouldn't want the stream to fill the buffer before returning. – arx Sep 26 '12 at 18:54
  • Is there any way to achieve buffering without writing my own BufferedStream layer? – axe Sep 26 '12 at 18:56
  • FileStream has its own buffer anyway. But what do you actually want the buffer for? – arx Sep 26 '12 at 19:05
  • For performance purposes. I'm going to scan the whole disk. Also my algorithm requires me to read portions of data based on some criteria. This portion For example I may need to read 4 bytes, so based on what you said I'd need to read data in chunks with size multiple to sector size. – axe Sep 26 '12 at 19:10
  • Actually, I'm not precisely sure why your 1000 byte buffer didn't work. [According to this](http://blogs.msdn.com/b/bclteam/archive/2004/10/29/249840.aspx) FileStream reads in 4096 byte chunks anyway. Maybe it doesn't buffer larger reads? Additionally, buffer addresses *might* need to be page-aligned in memory (second bullet point above). I don't know if FileStream does this. Ultimately, I'm not sure you can reliably read a volume using a FileStream. – arx Sep 26 '12 at 19:21
  • You are right. It doesn't use buffer for reads larger then buffer size. – axe Sep 26 '12 at 19:22
  • You might be able to use a [memory mapped file](http://blogs.msdn.com/b/bclteam/archive/2011/06/06/memory-mapped-file-quirks.aspx). This takes care of page-alignment and buffering, though it's a bit more of a pain to use (you'll have to update your view quite frequently). And I don't know if it will work with a volume. – arx Sep 26 '12 at 19:23
  • Regarding MMF. It throws invalid parameter exception when called for volume. Any suggestions on how to perform this? I just need to find some data on disk in raw mode. – axe Sep 26 '12 at 20:54
  • Is this production code? If not, I'd just continue to use FileStream with 4096 byte reads, which seems to be working. If you want smaller reads it looks like you would have to wrap your own buffer class around it. If this is production code you could pinvoke to VirtualAlloc (for appropriately aligned buffers) and ReadFile, and wrap this in a stream. This should work reliably. – arx Sep 26 '12 at 22:07
  • This is production code. Do you think it's not relibly to use 4096 byte reads? – axe Sep 27 '12 at 09:04
  • It's reliable in the sense that it should continue to work on a given computer if it works at all; your program won't fail halfway through a run. It's unreliable in the sense that you might find computers where it doesn't work, and it might be broken by future updates to Windows or .Net. The choice is yours. – arx Sep 27 '12 at 09:14
  • Isn't there any way to achive this in .NET without calling low level API? – axe Sep 27 '12 at 09:22
  • I tried to format drive with all possible allocation unit sizes and reading 4096 bytes works for each. I guess that's because doc says `Depending on the disk, this requirement may not be enforced`. – axe Sep 27 '12 at 09:27
  • I don't think there's a pure .Net solution because streams don't align the buffers and memory-mapped files don't work. Whether FileStream works or not is probably a function of the disk device driver rather than how the disk is formatted. – arx Sep 27 '12 at 09:38
  • Thanks. As far as I see those requirements only applies to the devices opened with FILE_FLAG_NO_BUFFERING flag. Where do you see that reads from volumes are non-buffered? – axe Sep 27 '12 at 09:43
  • The first paragraph I quoted in my answer "Volume handles can be opened as noncached (i.e. unbuffered) at the discretion of the particular file system, even when the noncached option is not specified in CreateFile." – arx Sep 27 '12 at 09:46
  • Still cannot find how to allocate aligned buffers. VirtualAlloc doesn't have corresponding parameters. – axe Sep 27 '12 at 10:50
  • VirtualAlloc allocates pages, which are 4K in size, so any allocation is at least 4K aligned. And it so happens [is actually 64K aligned](http://blogs.msdn.com/b/oldnewthing/archive/2003/10/08/55239.aspx). – arx Sep 27 '12 at 10:54
  • Interesting, couldn't find anything about that in MSDN. Thanks. – axe Sep 27 '12 at 10:59