How do I force collection of a MemoryStream's underlying buffer?

Question

I'm working with a memory stream and I'm running into some issues with some out of memory exceptions. The method causing the issue is below.

public override T Deserialize<T>(byte[] source)
{
    using (var stream = new MemoryStream(source))
    {
        var result = (T)_formatter.Deserialize(stream);

        return result;
    }
}

And this is a typical call to it:

var bufferSize = binaryArrays.Sum(x => x.Length);
var streamBuffer = new byte[bufferSize];

using (var stream = new MemoryStream(streamBuffer))
{
    foreach (var binaryArray in binaryArrays)
    {
        stream.Write(binaryArray, 0, binaryArray.Length);
    }

    result = serializer.Deserialize<T>(stream.ToArray());
}

I've implemented IDisposable in this method's containing class, and I'm disposing of the stream explicitly (even though it shouldn't be necessary) but I can't reclaim my memory. I understand that it is because the MemoryStream's underlying buffer is still floating around and that my application's virtual memory has been exhausted. So, how to kill the underlying buffer? Are there any techniques I could use here?

Much thanks.

[EDIT]

I fixed the using statement, but the issue is still present. Thanks for the catch.

Agree with @astander, the stream should be local and/or the code was damaged while posting. — H H, Sep 12 '13 at 14:02
The calling code sample seems wrong as well. Please be accurate, especially about how/where you declare buffers and resource objects. — H H, Sep 12 '13 at 14:05
@HenkHolterman: In particular, the "calling" code doesn't call `Deserialize` :) — Jon Skeet, Sep 12 '13 at 14:06
No, but I assume that object has the overridden Deserialize. It's unclear. — H H, Sep 12 '13 at 14:07

score 2 · Accepted Answer · answered Sep 12 '13 at 14:03

2

When you pass a byte array to the MemoryStream constructor, that byte array is the buffer. Perhaps you're still holding onto a reference to that byte array in the calling code?

The code you've shown won't hold onto the data (assuming it's actually allocating a local variable, rather than assigning to a property) - and disposing of the stream definitely isn't important here, and won't help at all... the data in a MemoryStream is available even after disposal anyway.

Of course, if the code you've given is really accurate, then I suggest you remove your Serializer and Stream properties/fields. It's very rarely a good idea to have using statements which reuse existing properties/variables rather than declaring new local variables.

using (Stream stream = new MemoryStream(source))
{
    return (T) _formatter.Deserializer(stream);
}

answered Sep 12 '13 at 14:03

Jon Skeet

1,421,763
867
9,128
9,194

So, will explicitly setting the incoming `byte[]` to `null` after it's been used take care of the issue? – elucid8 Sep 12 '13 at 14:06
@elucid8: Maybe, maybe not - we don't have *nearly* enough information about the situation to say, and you haven't even said where you would set it to null. Setting it to null in the `Deserialize` method certainly *wouldn't* have any effect. – Jon Skeet Sep 12 '13 at 14:08
Jon, as it turns out the byte array was only being held on to because of the serializer itself. I did as you suggested and it fixed the issue. Thanks for your help! – elucid8 Sep 12 '13 at 18:05

score 0 · Answer 2 · answered Sep 12 '13 at 14:03

This section does not seem right using (Stream = new MemoryStream(source)), where are you declaring Stream?

If you are declring this variable outside the using scope or method for that matter then

From using Statement (C# Reference)

You can instantiate the resource object and then pass the variable to the using statement, but this is not a best practice. In this case, the object remains in scope after control leaves the using block even though it will probably no longer have access to its unmanaged resources. In other words, it will no longer be fully initialized. If you try to use the object outside the using block, you risk causing an exception to be thrown. For this reason, it is generally better to instantiate the object in the using statement and limit its scope to the using block.

How do I force collection of a MemoryStream's underlying buffer?

2 Answers2