Memory issue when working with list of stream

Question

I'm working on an image processing project and I'm facing an issue.

I have a method that takes a stream (of an image) as input and returns a list of streams containng subparts of the provided image :

public List<System.IO.Stream> ImageProcessing(System.IO.Stream input){
    Bitmap inputAsBitmap = new Bitmap(input);
    // --- Applying miscellaneous filters ---

    List<Bitmap> subParts = GetSubParts(inputAsBitmap);
    inputAsBitmap.Dispose();

    List<System.IO.Stream> streams = new List<Stream>();
    for(int i = 0; i < subParts.Count; i++){
        MemoryStream memoryStream = new MemoryStream();
        subParts[i].Save(memoryStream, ImageFormat.Jpeg);
        streams.Add(memoryStream);
        subParts[i].Dispose();
    }
    return streams;
}

When I call this method manually by selecting an image through an OpenFileDialog everything works fine:

using (Stream imageStream = File.OpenRead(filename)){
    List<Stream> streams = ImageProcessing(imageStream);
    for(int i = 0; i < streams.Count; i++){
        System.Drawing.Image.FromStream(streams[i]).Save();
        streams[i].Dispose();
    }
}

But when I try to select a folder and call it in a for loop with every image belonging to it, an OutOfMemoryException is thrown:

for(int i = 0; i < filenames.Count; i++){
    using (Stream imageStream = File.OpenRead(filenames[i])){
        List<Stream> streams = ImageProcessing(imageStream);
        for(int j = 0; j < streams.Count; j++){
            System.Drawing.Image.FromStream(streams[j]).Save();
            streams[j].Dispose();
        }
    }
}

While investigating, I found that having a block of instructions that takes around 20ms to execute allows the memory to get cleaned and thus prevents the exception to be thrown:

for(int i = 0; i < filenames.Count; i++){
    // --- A block of instructions placed here will prevent the exception ---
    using (Stream imageStream = File.OpenRead(filenames[i])){
        // -- Same code as above ---
    }
}

Moreover, while checking the memory on several points (by using Process.GetCurrentProcess().PrivateMemorySize64 or System.GC.GetTotalMemory()) it seems that disposing of Bitmap and Stream does not change anything concerning the amount of used memory.

Any hint on how I could get rid of this OutOfMemoryException?

Do you have any explanation concerning the fact that the memory does not get cleaned when calling the ImageProcessing method in a for loop while it does between two calls when they are done manually?

---

EDIT:

This is what I have now, but still not working:

for(int i = 0; i < filenames.Count; i++){
    using (Stream imageStream = File.OpenRead(filenames[i])){
        List<Stream> streams = ImageProcessing(imageStream);
        for(int j = 0; j < streams.Count; j++){
            using (var image = System.Drawing.Image.FromStream(streams[j]))
                image.Save();
            streams[j] = null;
            //System.GC.Collect();
        }
    }
}
//System.GC.Collect();

Answer 1

There are few things that you can do to improve things.

First, you should force the application to be run as a 64 bits application and run the program in Release mode as the Debug mode may cause the program to hold on instances a bit longer.

Then the call to System.Drawing.Image.FromStream(streams[j]).Save(); returns an Image so it must be disposed as well.

Finally, I am not a fan of putting null when I am done with some variable. The JIT / GC usually knows when a variable is not used anymore. However, in your case, the stream[j] instances are being kept in the list during the whole process. Maybe you could try changing the structure of your program a bit with the use of IEnumerable and yield return so that the instances are not stuck in a list for too long. Here are the key points:

public IEnumerable<Bitmap> GetSubParts()
{
   ...
   // Assuming there is a loop somewhere:
   for(***)
   { 
      yield return ** Bitmap
   }
}

Same with ImageProcessing:

public IEnumerable<System.IO.Stream> ImageProcessing(System.IO.Stream input)
{
    using(Bitmap inputAsBitmap = new Bitmap(input))
    {
        // --- Applying miscellaneous filters ---

        foreach(var subPart in GetSubParts(inputAsBitmap))
        {
            MemoryStream memoryStream = new MemoryStream();

            using(subPart)
            {
                subPart.Save(memoryStream, ImageFormat.Jpeg);
            }

            yield return memoryStream;
        }
    }
}

And finally:

for(int i = 0; i < filenames.Count; i++)
{
    using (Stream imageStream = File.OpenRead(filenames[i]))
    {
        foreach(var stream = ImageProcessing(imageStream))
        {
            using(stream)
            {        
                 using(var image = System.Drawing.Image.FromStream(stream))
                     image.Save();
            }
        }
    }
}