How to fix out of memory error in a recursive method C#

Question

My problm is that I have a recursive method that given a path it enter in all the subFolders and store the file names and some info in the database. It worked fine until I used a folder that has 31000 files and a lot of subfolders. My parameters to the method is the the path that he is working, Parent Path as a database class and a database class to perform some inserts and selects. My question is if by using the ref keyword it would help about the memory issue?

Sorry if there is an article about this. I didn't found anything.

public void ManageFiles(string path, Table Father, DataBaseHandlerClass DB)
{
    try
    {
        log.Info("---------------------------Inicio Pasta " + path + "--------------------------");
        DirectoryInfo d = new DirectoryInfo(path);
        DirectoryInfo[] Dirs = d.GetDirectories();
        int hashCode = path.GetHashCode();
        IFB_FilePath FilePath = DB.InsertOrUpdatePath(hashCode, path, Father == null ? null : (int?)Father.ID);

        foreach (var newPath in Dirs)
        {
            ManageFiles(newPath.FullName, FilePath, DB);
        }

        List<string> ext = DB.GetExtensionsToSearch();

        FileInfo[] Files = d.GetFiles("*.*").Where(s => ext.Contains(s.Extension.ToLower())).ToArray();
        var watch = System.Diagnostics.Stopwatch.StartNew();
        foreach (FileInfo file in Files)
        {
            watch = System.Diagnostics.Stopwatch.StartNew();
            FileTable Ficheiro = DB.InsertOrUpdateFile(file, FilePath);
            if (Ficheiro.ID_Extension_Group == 1 && !Ficheiro.Metadata.Any())
            {
                CreateFileMetadata(DB, Ficheiro);
            }
            else
            {
                if (Ficheiro.ID_Extension_Group == 1 && Ficheiro.TakenDate == null)
                {
                    UpdateTakenDate(DB, Ficheiro);
                }
            }

            watch.Stop();

            log.Info("Tempo em ms: " + watch.ElapsedMilliseconds + " Ficheiro " + file.Name);
        }
        log.Info("---------------------------Fim Pasta " + path + "--------------------------");
    }
    catch (Exception Ex)
    {
        log.Error(Ex);
    }
}

I Think that i'm gonna firsst sstore all the Path locations in the databae and then one by one im gonna watch the files. — TC Gaming, Aug 01 '19 at 09:23

score 3 · Accepted Answer · answered Aug 01 '19 at 09:41

3

Although recursion is beautiful and sometimes requires less code, i suggest you to unwrap it into cycle. C# have recursion depth, and then it exceeded, runtime throws error.

But if rewriting code is not an option, you can acquire bigger limit of recursion.

As this site says, nesting in C# can be as big, as 14250 calls. So in your case, calling function 31000 times exceeds this limit.

answered Aug 01 '19 at 09:41

nonForgivingJesus

595
4
15

Thanks for you answer. I would love to not rewrite it but i think it is the best thing to do. Im going save the code so maybe someone has another idea, and im going to work on rewriting it. – TC Gaming Aug 01 '19 at 09:49

score 0 · Answer 2 · answered Aug 01 '19 at 14:30

0

I think the first thing you should move the following lines outside your ManageFiles function:

 DirectoryInfo d = new DirectoryInfo(path);
 DirectoryInfo[] Dirs = d.GetDirectories();
 int hashCode = path.GetHashCode();

This is because you are creating a new DirectoryInfo object and the collection of directories with each call to the function. This is the first change i would make as it would certainly have an impact on memory.

answered Aug 01 '19 at 14:30

Nick

3,454
6
33
56

thank you for you answe but I need to get the info of the subfolders. I made a change and use the recursive method only for the folders and insert them into the database. Later at night I will execute it and then see if the error continues. – TC Gaming Aug 01 '19 at 15:10
I did some changes in the DBContext too and now I'm disposing the context after using it. – TC Gaming Aug 01 '19 at 15:12

How to fix out of memory error in a recursive method C#

2 Answers2