Reading pictures from DB using memorystreams and images often crashes with Out of Memory

Question

I'm trying to figure out if there is something seriously wrong with the following code. It reads the binary from the database, stores it as a picture and associates with an object of an Animal record.

For each row (record of an animal):

byte[] ba = (byte[])x.ItemArray[1]; //reading binary from a DB row
using (MemoryStream m=new MemoryStream(ba))
 {
    Image i = Image.FromStream(m);  //exception thrown occassionally
    c.Photo = i;
    listOfAnimals.Add(c);
 }

First of all, with 18 pictures loaded (the JPG files have 105 Mb in total), the running app uses 2 gb of memory. With no pictures loaded, it is only 500 Mb. Often the exception gets raised in the marked point, the source of which is System Drawing.

Could anyone help me optimize the code or tell me what the problem is? I must have used some wrong functions...

You have run out of memory! the only way of solving is to add more memory or use your hard drive as swap space when memory is exhausted. — jdweng, Jul 28 '21 at 09:07
@jdweng But how can 105 Mb of pictures occupy and deplete 2 Gb of memory? I suspect it is not being cleared out properly — John V, Jul 28 '21 at 09:08
A JPEG is compressed, so the size of the JPG file != amount of memory required. An image may require up to 4 bytes per pixel. So an 8MP image may only take 1 - 5 MB disk space (depending on compression level) but could require 24 MB of memory. — jason.kaisersmith, Jul 28 '21 at 09:10
@jason.kaisersmith Good point, I completely missed this fact! It would still not explain the situation, though — John V, Jul 28 '21 at 09:13
Don't `using..` to create the `MemoryStream`. It should be alive as long as the image is. `Image i = Image.FromStream(new MemoryStream(ba))`. — dr.null, Jul 28 '21 at 09:14
@dr.null Well I thought I should use it only to create the Image out of the binary data. — John V, Jul 28 '21 at 09:16
Yes but it is the _soul_ of the image here I'd say. Please read the [Remarks](https://learn.microsoft.com/en-us/dotnet/api/system.drawing.image.fromstream?view=net-5.0#System_Drawing_Image_FromStream_System_IO_Stream_) section. — dr.null, Jul 28 '21 at 09:19
Basic tip - don't store binary files in a database. Can you store them on a file system instead? — mjwills, Jul 28 '21 at 09:26
`the JPG files have 105 Mb in total` 105MB or 105Mb (bytes, or bits)? What is the resolution of each of these files? Why are you showing 18 high resolution images at the same time? — mjwills, Jul 28 '21 at 09:28
Forgot to mention, don't forget to dispose of the images when they are no longer needed, before loading a new set, or when you close the `Form`. — dr.null, Jul 28 '21 at 09:31
I do not consider this a memory leak yet since you are still creating the object. Memory leaks are after you used the object and want to dispose. When creating an application may use additional resource that you cannot control — jdweng, Jul 28 '21 at 14:28

JonasH · Answer 1 · 2021-07-28T13:50:35.213

Checking the documentation, a possible reason for out of memory exceptions are that the stream is not a valid image. If this is the case it should fail reliably for a given image, so check if any particular source image is causing this issue.

Another possibility should be that you simply run out of memory. Jpeg typically gets a 10:1 compression level, so 105Mib of compressed data could use +1Gib of memory. I would recommend switching to x64 if at all possible, I see be little reason not to do so today.

There could also be a memory leak, the best way to investigate this would be with a memory profiler. This might be in just about any part of your code, so it is difficult to know without profiling.

You might also need to care about memory fragmentation. Large datablocks are stored in the large object heap, and this is not automatically defragmented. So after running a while you might still have memory available, just not in any continuous block. Again, switching to x64 would mostly solve this problem.

Also, as mjwills comments, please do not store large files in the database. I just spent several hours recovering a huge database, something that would have been much faster if images where stored as files instead.

Tu deschizi eu inchid · Accepted Answer · 2021-07-28T21:56:13.173

According to Image.FromStream Method

OutOfMemoryException

The stream does not have a valid image format.

Remarks You must keep the stream open for the lifetime of the Image.

The stream is reset to zero if this method is called successively with the same stream.

For more information see: Loading an image from a stream without keeping the stream open and Returning Image using Image.FromStream

Try the following:

Create a method to convert byte[] to image

ConvertByteArrayToImage

public static Image ConvertByteArrayToImage(byte[] buffer) 
{
    using (MemoryStream ms = new MemoryStream(buffer))
    {
        return Image.FromStream(ms);
    }
}

Then:

byte[] ba = (byte[])x.ItemArray[1]; //reading binary from a DB row
c.Photo = ConvertByteArrayToImage(ba);
listOfAnimals.Add(c);

Reading pictures from DB using memorystreams and images often crashes with Out of Memory

2 Answers2

Linked