GetFiles is incredibly slow vb.NET

Question

I need to process a large amount of files and scour thru a group of directories and subdirectories and get the file from the found directory. This is easy using a simple directory but when scouring a thousand directories, the program slows down to a crawl. I've heard of using enumeratefiles but I'm not how to approach this. thank you.

How should I approach this?

 dim Filename_to_lookfor  as string 
 dim  filePaths As String()

for   i = 0 to 1000 
   Filename_to_lookfor = array(i)

   filePaths = Directory.GetFiles(sTargetPath, sFilename_to_lookfor.ToUpper, 
   IO.SearchOption.AllDirectories)

   if filepath.length > 0 then 
    'copy file using Computer.FileSystem.CopyFile()
   end if

 next

Thank you.

How long does it take a spacecraft to get to Mars? Does that mean that it's slow? No, it just means that it has a long way to go. `GetFiles` is not slow. It's always the same speed. If it has a lot of files to process then of course that takes a long time, but that's not slow. Don't do anything that takes a long time on the UI thread because then it will be too busy to maintain the UI. If it's a WinForms app then try using a `BackgroundWorker`. That's probably the simplest option when you're starting out. — , Jan 29 '22 at 09:21
`GetFiles` will run to completion first and then return all the files in an array. `EnumerateFiles` allows you to process the files one by one, as they are retrieved. That can be good if you might want to stop part-way through or the number of files is really large. If you want to get all the files before doing anything with them though, `GetFiles` is the better choice. If you want to use `EnumerateFiles` then just call it and loop over the result. It's that simple. Don't try to make it complex when it's not. — , Jan 29 '22 at 09:23

Xavier Junqué · Answer 1 · 2022-01-29T20:45:04.907

Another coding would be using Linq:

Sub SearchFiles(pathAndfileNames_to_lookfor() As String, sTargetPath As String)

    ' Call just once Directory.GetFiles():
    Dim FilePaths() As String = Directory.GetFiles(sTargetPath, "*.*", IO.SearchOption.AllDirectories)

    Dim lookup1 = FilePaths.ToLookup(Function(x) x.ToLower)
    Dim lookup2 = pathAndfileNames_to_lookfor.ToLookup(Function(x) x.ToLower)
    Dim filesInBoth = lookup1.SelectMany(Function(x) x.Take(lookup2(x.Key).Count).ToArray)
    For Each file In filesInBoth
        'copy file using Computer.FileSystem.CopyFile()
        Console.WriteLine(file)
    Next
End Sub

Calling the procedure would be as follows:

    Dim file1 As String = "C:\...\file1..."
    Dim file2 As String = "C:\...\file2..."

    Dim pathAndfileNames_to_lookfor() As String = New String() {file1, file2}
    Dim sTargetPath = "..."
    SearchFiles(pathAndfileNames_to_lookfor, sTargetPath)

GetFiles is incredibly slow vb.NET

1 Answers1