C# .Net Freeze while iterating through large number of files

Question

I have a problem that I wrote an application that would iterate through files and add +1 to the integer each file, until it reaches a specific file name. The problem is probably because .Net does not access the native file system directly, it fills up collections, but in my case it would take years, believe me, I have 260 000 files in the target folder. The iteration does not even reach the second file. The thread just totally freezes, no errors, no exceptions. So is there any way to get a direct access to the Native File System without any useless collection filling ups?

Here is my code:

private void button1_Click(object sender, EventArgs e)
{
    try
    {
        foreach (string file in Directory.GetFiles("\\\\Mypcname-PC\\vxheaven\\malware"))
        {
            count++;
            label1.Text = Convert.ToString(count);
            if (file.Contains(textBox1.Text))
            {
                label1.Text = Convert.ToString(count) + " reached the file";
                break;
            }
        }
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
    }
}

btw. Sorry for my bad english

Regards

If you want us to help you with your code you need to show your code. [Edit your question](http://stackoverflow.com/posts/38083668/edit) and include the code that itterates the files. — Scott Chamberlain, Jun 28 '16 at 18:29
Sounds like a problem in the code if it never gets to the second element. Possibly because it sounds like you may be modifying the collection — Ňɏssa Pøngjǣrdenlarp, Jun 28 '16 at 18:29
That is not enough information, We need to see a [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve) or your question will be closed. — Scott Chamberlain, Jun 28 '16 at 18:31
this should take second or two. measure how long it takes to fetch file names from network location. that looks like possible candidate for bottlenck — Tigran, Jun 28 '16 at 18:35
Sorry, but what does bottlenck mean? (as i mentioned im not English) — Yogibear, Jun 28 '16 at 18:36
Your UI thread is getting locked up here. You should perform your operation on a thread-safe BackgroundWorker/Task and update the UI based on the progress returned by the worker/task. — ManoDestra, Jun 28 '16 at 18:39

Scott Chamberlain · Accepted Answer · 2016-06-28T18:38:35.127

10

Because you are doing all the work on the UI thread it can't refresh while it is working. You need to do the work on a background thread then update the UI in a thread safe way. Also switching to Directory.EnumerateFiles will make it faster to read the first file so it does not need to store all the records in to an array. Lastly I changed ex.Message to ex.ToString(), it will display much more useful information that way.

private async void button1_Click(object sender, EventArgs e)
{
    try
    {
         var text = textBox1.Text;
         var progress = new Progress<string>((x) => label1.Text = x);
         await Task.Run(() => DoWork(progress, text));
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
}

private void DoWork(IProgress<string> progress, string text)
{
    foreach (string file in Directory.EnumerateFiles("\\\\Mypcname-PC\\vxheaven\\malware"))
    {
        count++;
        progress.Report(Convert.ToString(count));
        if (file.Contains(text))
        {
            progress.Report(Convert.ToString(count) + " reached the file");
            break;
        }
    }
}

(Code was written in a web browser from memory so there may be errors)

edited Jun 28 '16 at 18:38

answered Jun 28 '16 at 18:36

Scott Chamberlain

124,994
33
282
431

Tought about that, but I didn't think that this would be the problem, I'll check this out – Yogibear Jun 28 '16 at 18:38
2

The reason it looks like it is not working is because you are blocking the UI, it is working but it just can't tell you that it is working – Scott Chamberlain Jun 28 '16 at 18:40
Okay, copied into project, the only question is, what additional references should I include? – Yogibear Jun 28 '16 at 18:40
Add `using System.Threading.Tasks;` to the top of the file. Also, this code requires .NET 4.5 or newer. – Scott Chamberlain Jun 28 '16 at 18:41
Okay, I'll set the project to .Net 4.5 – Yogibear Jun 28 '16 at 18:42
Works like charm, thanks, didn't think UI thread would be the problem – Yogibear Jun 28 '16 at 18:47

score -1 · Answer 2 · answered Jun 28 '16 at 18:41

Seems like you are using a potentially very time-consuming loop without ever processing the Windows message queue, therefore your application may APPEAR to be frozen, while it's probably just busy doing what you instructed it to do in the loop. Try this:

private void button1_Click(object sender, EventArgs e)
{
    try
    {
        foreach (string file in Directory.GetFiles("\\\\Mypcname-PC\\vxheaven\\malware"))
        {
            count++;
            label1.Text = Convert.ToString(count);
            Application.DoEvents();
            if (file.Contains(textBox1.Text))
            {
                label1.Text = Convert.ToString(count) + " reached the file";
                break;
            }
        }
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
    }
}

`Application.DoEvents();` is very dangerous to do, you can screw up the internal state of your program if you are not careful. You should not use it as your first choice of options. — Scott Chamberlain, Jun 28 '16 at 18:43
That was a quick and dirty fix without changing his original code. Of course your code is a lot better (voting up your answer) but it's a complete rewrite, not a fix. ;) — FjodrSo, Jun 28 '16 at 18:48

C# .Net Freeze while iterating through large number of files

2 Answers2

Linked