0

I wrote a small program in VB .Net to physically sort folder entries on FAT partitions (see http://www.public.bplaced.net). To do this, I use API calls to Kernel32.dll. I open the partition with CreateFileA, use the handle to lock the drive with DeviceIoControl FSCTL_LOCK_VOLUME and then read and write sectors directly with SetFilePointer / ReadFile / WriteFile (synchronous, with NO_BUFFERING).

All works PERFECTLY, except when I use my program with partitions larger than 2147483647 sectors (7FFF FFFF hex), WriteFile reports success but returns (and has) 0 byte written. This is the case no matter which sector I try to write. EVERYTHING else seems to still work as it should (including correct sector reading). When I use PartitionWizard to shrink the partition below the above limit, writing works again.

Question: does ANYBODY have a clue what could cause this strange behavior? My wild guess is that 'something' might interpret a value greater than 7FFF FFFF as 'signed'? Not within my code, the 'total sectors of partition' is not needed anywhere.

A friend also said that when he did something similar with 'streams', writing worked even with a 'large' partition...

I'm a total N00b, I can't even memorize all the terminology (but I still want to program so dearly...), so if you might have an explanation / hint / whatever, please describe it as simple-worded and detailled as possible.

Some code snippets (don't know where to start)... program is compiled for x86 systems. Problem occurs on both Win7 x86 and Win7 x64.

<DllImport("kernel32.dll", ExactSpelling:=True, SetLastError:=True, CharSet:=CharSet.Auto)> _
Public Shared Function DeviceIoControl _
         ( _
         ByVal hDevice As IntPtr, _
         ByVal dwIoControlCode As UInteger, _
         ByVal lpInBuffer As IntPtr, _
         ByVal nInBufferSize As UInteger, _
         ByVal lpOutBuffer As IntPtr, _
         ByVal nOutBufferSize As UInteger, _
         ByRef lpBytesReturned As UInteger, _
         ByVal lpOverlapped As IntPtr _
         ) _
         As Integer
End Function

Public Declare Function CreateFile Lib "kernel32" _
    Alias "CreateFileA" ( _
                        ByVal lpFileName As String, _
                        ByVal dwDesiredAccess As Int32, _
                        ByVal dwShareMode As Int32, _
                        ByVal lpSecurityAttributes As IntPtr, _
                        ByVal dwCreationDistribution As Int32, _
                        ByVal dwFlagsAndAttributes As Int32, _
                        ByVal hTemplateFile As Int32 _
                        ) _
                        As IntPtr

Public Declare Function SetFilePointer Lib "kernel32" _
                    ( _
                    ByVal hFile As IntPtr, _
                    ByVal lpDistanceToMove As UInt32, _
                    ByRef lpDistanceToMoveHigh As Int32, _
                    ByVal dwMoveMethod As UInt32 _
                    ) _
                    As UInt32

Public Declare Function WriteFile Lib "kernel32" _
                        ( _
                        ByVal hFile As IntPtr, _
                        ByVal lpBuffer As Byte(), _
                        ByVal nNumberOfBytesToWrite As Int32, _
                        ByRef lpNumberOfBytesWritten As Int32, _
                        ByVal lpOverlapped As IntPtr _
                        ) _
                        As Boolean

' **********************************************************

' open the partition by drive letter

    devicehandle = CreateFile( _
                             "\\.\" & driveletter & ":", _
                             GENERIC_READ Or GENERIC_WRITE, _
                             FILE_SHARE_READ Or FILE_SHARE_WRITE, _
                             IntPtr.Zero, _
                             OPEN_EXISTING, _
                             FILE_ATTRIBUTE_NORMAL Or FILE_FLAG_NO_BUFFERING, _
                             0 _
                             )

' **********************************************************

' lock the partition

    Dim unused_lv As UInteger

    Dim locked As Integer = DeviceIoControl( _
                                           devicehandle, _
                                           FSCTL_LOCK_VOLUME, _
                                           IntPtr.Zero, _
                                           0, _
                                           IntPtr.Zero, _
                                           0, _
                                           unused_lv, _
                                           IntPtr.Zero _
                                           )

' **********************************************************

' set the file pointer, sector = sector to read, bytes_per_sector = 512. I use Bitconverter to get the hi and lo DWORDs

    Dim s_bytes() As Byte = BitConverter.GetBytes(sector * bytes_per_sector)
    ' Hi-DWORD
    Dim byte_dist_high As Int32 = BitConverter.ToInt32(s_bytes, 4)   ' byte 4 - 7
    ' Lo-DWORD
    Dim byte_dist_low As UInt32 = BitConverter.ToUInt32(s_bytes, 0)  ' byte 0 - 3

    ' move file pointer
    Dim move As UInt32 = SetFilePointer( _
                         devicehandle, _
                         byte_dist_low, _
                         byte_dist_high, _
                         FILE_BEGIN _
                         )

' **********************************************************

    ' write a sector
    Dim write As Boolean = WriteFile( _
                                    devicehandle, _
                                    buffer, _
                                    bytes_per_sector, _
                                    bytes_written, _
                                    IntPtr.Zero _
                                    )

    If write = False Then
        Return False
    Else
        Return True
    End If
N00b
  • 29
  • 5
  • 1
    Defect in your code. You didn't show any. [mcve] – David Heffernan Jul 15 '17 at 08:31
  • Since the code works perfectly with all 'smaller' partitions, I really don't know what to show, that's why I _described_ what I did instead. What kind of defect in my code could this be, where reading succeeds, but writing fails but reports success? ReadFile and WriteFile both use the same FilePointer settings and parameter. What do you suggest should I show? – N00b Jul 15 '17 at 08:42
  • Since you are calling [SetFilePointer](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365541.aspx), I'm assuming you are P/Invoking it. This requires a correct API declaration, and maybe you got that wrong. With [pinvoke.net](http://www.pinvoke.net/default.aspx/kernel32/SetFilePointer.html) being such a useless (and popular) resource, that is indeed likely. – IInspectable Jul 15 '17 at 09:37
  • Show a [mcve] please – David Heffernan Jul 15 '17 at 09:37
  • @IInspectable : I agree with that pinvoke.net is rather useless for VB (since almost all those declarations are wrong and/or for VB6), but _most_ C# versions are correct. I usually take the C# version and run it through an online converter (if I don't decide to write the P/Invoke declaration myself). – Visual Vincent Jul 15 '17 at 10:49
  • 1
    @VisualVincent: Very few of the C# signatures are correct as well. Last time I bothered to check (about 3 years ago), hardly any of the signatures would work for 64-bit code. And the signature I linked to in my previous comment is incomplete, at best. There simply is no reason to omit the `SetLastError=true` attribute. As it stands, pinvoke.net is as useless as it ever was. – IInspectable Jul 15 '17 at 14:07
  • @IInspectable : I guess we use a different set of methods, at least _most of the ones I've used_ have been more or less correct. I can write the declaration myself but I am often too lazy to do so. ;p – Visual Vincent Jul 15 '17 at 14:41
  • As I said, since READING of sectors (with SetFilePointer / ReadFile) works with all partition sizes, there is no reason why WRITING (with SetFilePointer / WriteFile) should not succeed on large partitions. How do I show some code? The messages here seem to have a character size limit... – N00b Jul 15 '17 at 15:05
  • There's an [edit](https://stackoverflow.com/posts/45116162/edit) link at the bottom of your question. Questions do not have a character size limit. – IInspectable Jul 15 '17 at 15:19
  • Where is your error handling code? How do you determine, that an API call failed, and which error code it returned? – IInspectable Jul 15 '17 at 15:46
  • Sorry, but the problem can't be in my error handling. When WriteFile returns boolean TRUE then it should have succeeded, otherwise FALSE. In my case, on partitions with more than 7FFF FFF sectors, WriteFile returns TRUE whenever I write, but bytes_written shows 0 byte instead of 512, and no writing was done... If write = False Then Return False Else Return True End If And If I need the errorcode I use Marshal.GetLastWin32Error() – N00b Jul 15 '17 at 16:03
  • How did you determine the value for `bytes_per_sector`, are you sure the device in question uses 512-byte sectors? Also, is `sector` a 64-bit integer? – Harry Johnston Jul 16 '17 at 01:30
  • sector is an Int64, and I get bytes_per_sector by reading the sector size value from the Partition Boot Record. I do know about disks with larger *physical* sectors (e.g. 4096), but the ones I tested it with do 512e sector emulation. And *since it ALL works fine when I just shrink the partition below the 80000000 hex sector count, I STRONGLY doubt that writing fails because of this*. REALLY, code is probably fine, ESPECIALLY because the total sector size is nowhere needed in the program, and reading/writing is all done very much at the beginning. Even when I try writing sector 0 it fails. – N00b Jul 16 '17 at 06:33
  • Question is: why does *reading* work correctly but *writing* (using same File Pointer setting and same parameters and all) fails when the partition is larger than 2147483647 sectors (7FFFFFFF hex)? This does not sound like it's a problem in my program, more likely that an API function might do something wrong (even though I can't really believe it)... – N00b Jul 16 '17 at 06:42
  • IMO, a Windows bug isn't out of the question in this particular case. What version(s) of Windows are exhibiting the problem? – Harry Johnston Jul 16 '17 at 07:53
  • I tested with Win7 x86 and x64 ... program is compiled for x86 versions – N00b Jul 16 '17 at 07:56
  • Does the problem occur on NTFS partitions? Or only FAT? (I didn't think FAT even *supported* partitions that size ... are you doing anything special to create them?) – Harry Johnston Jul 17 '17 at 05:14
  • I think you used the wrong type (is:uinteger > should be: long). As already stated check pinvoke or msdn for the correct parameter types – David Sdot Jul 17 '17 at 11:25
  • I did not test on NTFS partitions, but would it make a difference when I read the data directly anyway? You found a bug? (use of wrong type) Then please tell me *exactly* what to change, and where. As far as I know, LONG is a signed 64bit value, so INT64 is probably the equivalent. And where the 64bit have to be split up into two 32bit values (e.g. for SetPointer) I used an UINT32 (lo) and INT32 (hi). When I wrote the program, I spent an incredible time at PINVOKE and other places to get the API calls right, *if you see a confirmed error in my calls anywhere PLEASE explain it to me*. – N00b Jul 17 '17 at 12:24
  • @N00b, I'm not certain, but I *think* that since you are opening the specific volume rather than the disk as a whole the file system could affect the way the request is handled. But it's a moot point now as far as I'm concerned, since I've realized none of my test machines have disks that big, so I'm not going to be able to reproduce the problem anyway. It might help attract interest to the question if you could post a [mcve], e.g., a simple program that reads sector zero, makes a change (e.g., the volume serial number) and tries to write it back. – Harry Johnston Jul 21 '17 at 00:06
  • @Harry Johnston - Ack! You are correct, on the exact same partition in NTFS, writing succeeds. But how can this be? I created the FAT partition with Partition Wizard, and technically, the maximum possible size should be 8TB. How do I post a complete example? Just copy the code in here? – N00b Jul 21 '17 at 13:08
  • Edit it into your question. It would probably be reasonable for the complete example to replace the existing code snippets, no need for both - though of course that's up to you to decide. – Harry Johnston Jul 22 '17 at 01:13
  • My *guess* is that the Windows 7 FAT driver was designed with an assumption that no FAT volumes would be over 1TB in size, or that it was never tested on such volumes. I'm not sure whether Microsoft would consider this a bug or not, and it might or might not be the same in later versions of Windows. (There might even be a hotfix.) – Harry Johnston Jul 22 '17 at 01:48
  • You may have to work around the problem by figuring out which physical disk the volume is on, and what the offset from the start of disk to the volume is, and using, e.g., `\\.\PhysicalDrive0` instead of `\\.\E:` - I'm not sure off the top of my head what the best way to get the necessary information is. – Harry Johnston Jul 22 '17 at 01:49
  • I just had the same idea, but don't quite know how to find the first _physical_ sector of a partition on a volume without going through the partition tables manually... that's why I used \\.\E: to open a partition, not the \\.\PhysicalDrive stuff. Do _you_ have any ideas for easy implementation? Can CreateFile be used with a 'different' driver, that reads 'just sectors' regardless of the underlying file system? – N00b Jul 22 '17 at 01:53

0 Answers0