5

I have a loop in my code that generates many byte[] arrays (around 1 to 2 MB each), fills them with data, and then discard the reference. So, even though the reference is only held for a short time, I can see the private working set growing.

Now, if I try to allocate a large array (~ 400 MB) after the loop, could I get an out of memory exception? Or will the allocation force the GC to collect the transient data?

Thanks!

Jacko
  • 12,665
  • 18
  • 75
  • 126

3 Answers3

2

Generating many 1-2MB arrays is a bad idea. Even if you avoid out of memory situation the performance really suffers. Allocating many short lived objects on the large object heap is an allocation pattern the current GC doesn't handle well.

I strongly recommend recycling them whenever possible. Implement a pool into which you throw the arrays once you don't need them anymore. And then when allocating first check if you can satisfy the request from the pool. That pattern resulted in huge performance benefits in one of my programs.

I think full memory forces a GC, but if unmanaged allocations happen around the same time you still can get OOMs.

CodesInChaos
  • 106,488
  • 23
  • 218
  • 262
  • 1
    "Allocating many short lived objects on the large object heap is an allocation pattern the current GC doesn't handle well." Care to site your source here? – Andy May 18 '11 at 19:21
  • @Andy That's my own experience. I had a project that took several screenshots into arrays per second, and the performance got much better once I used pooling to reuse arrays. – CodesInChaos May 20 '11 at 08:25
  • I assume the reason is simply that it's the high amount of memory per second you allocate which forces GCs all the time. With large objects you allocate and discard large amount of memories even with relatively few allocations per second. And I read somewhere that LOH objects are only collected during a Gen2 collection, but I'm not sure about that. – CodesInChaos May 20 '11 at 08:27
1

If you are worried about it, you can always call GC.Collect(); before that large array after the loop, that will force a garbage collection of all generations. Don't do in in the loop however, unless you are not concerned about time as this can be rather slow (too slow for a loop, not so much for a one off thing.)

Archimagus
  • 11
  • 1
0

This really depends. You cannot be sure that the garbage collector will discard in time. With byte arrays you are reasonably safe, but most objects are discarded too late if you make heavy use of them without using the dispose() method.
This results in out-of-memory exceptions even though you might have discarded all references.
If you are experience problems, you could try GC.Collect(0);, although this usually is ill-advisable.

Mr47
  • 2,655
  • 1
  • 19
  • 25
  • Dispose has _nothing_ to do with GC. – H H May 18 '11 at 18:26
  • How does dispose help here? I don't think byte arrays even have it. – CodesInChaos May 18 '11 at 18:27
  • Dispose doesn't actually free memory. It usually releases *unmanaged* resources. – Jon Skeet May 18 '11 at 18:27
  • If I am not mistaken, using dispose marks the object as finalized, resulting in the finalizer not having to run when garbage is collected. This allows the GC to free up memory more times with less effort. – Mr47 May 18 '11 at 18:28
  • Even if that's correct, you don't have to worry about this unless perhaps in very special circumstances. –  May 18 '11 at 18:34
  • @Mr47 by convention Dispose suppresses finalization but if there is no finalizer declared this doesn't have any effect. Only objects that directly manage unmanaged resources should have a finalizer so for the most part dispose isn't going to help with memory in managed heaps. – Yaur May 19 '11 at 01:04