6

I have massive directories, and I would like to read all the files as fast as I can. I mean, not DirectoryInfo.GetFiles fast, but 'get-clusters-from-disk-low-level' fast.

Of course, .NET 2.0, c#

Similar question was here, but this approach wasn't any good:

C# Directory listing massive directory

Someone suggested pInvoke on FindFirst/FindNext. Anybody tried that and is able to share results?

Community
  • 1
  • 1
Daniel Mošmondor
  • 19,718
  • 12
  • 58
  • 99
  • Using reflector it looks like DirectoryInfo.GetFiles ultimately boils down to FindFirstFile/FindNextFile kernel32 calls anyway. – Daniel Renshaw Aug 27 '10 at 09:36

3 Answers3

5

For a "normal" approach, basically everything boils down to FindFirstFile/FindNextFile, you don't really get much faster than that... and that isn't super-turbo-fast.

If you really need speed, look into reading the MFT manually - but know that this requires admin privileges, and is prone to break whenever NTFS gets updated (and, oh yeah, won't work for non-NTFS filesystems). You might want to have a look at this code which has USN and MFT stuff.

However, perhaps there's a different solution. If your app is running constantly and needs to pick up changes, you can start off by doing one slow FindFirstFile/FindNextFile pass, and then use directory change notification support to be informed of updates... that works for limited users, and doesn't depend on filesystem structures.

snemarch
  • 4,958
  • 26
  • 38
3

For the best performance, it is possible to P/Invoke NtQueryDirectoryFile, documented as ZwQueryDirectoryFile.

(That short of accessing the disk directly and reading the raw file system structures directly, which usually is not practical.)

nick.lowe
  • 199
  • 1
  • 5
1

Try using something like this DirectoryManager and refine it by your needs. Works faster than the .NET Framework GetDirectories() or GetFiles() because we ommitted there cross-platform checkings and adaptations.

Eugene Cheverda
  • 8,760
  • 2
  • 33
  • 18
  • 2
    Very nice little class, it uses `FindFirstFile/FindNextFile`. Personally I would add a copy of the code here instead of linking to another site, just in case that one goes down. – Thymine Mar 22 '12 at 22:22
  • 6
    @Eugene Cheverda The DirectoryManager link no longer works :P – Dave Gordon Feb 04 '14 at 23:32