who profits more from pooling? managed / unmanaged?

Question

Having two similar applications: one managed and the other unmanged. Both utilizing a heavy allocation pattern for large objects. I.e. they are requesting a lot of those objects in (long running) loop and both releasing those objects pretty right away after usage. The managed app uses IDisposable() which gets called immediately. The unmanaged is utilizing the destructors.

Some but not all of the objects could be reused. So, a pool is considered in order to increase execution speed and minimize the risk of memory fragmentation.

Which application would you expect to profit more from a pooling and why?

@Update: This is about a mathematical library. Therefore, those large objects will be arrays of value type. Mostly large enough for LOH. I am positive, pooling would improve performance on the managed side a lot. Many libraries exist - for managed/unmanaged environments. None of them I know does such pooling really. I am wondering why?

Answer 1

First, a little consideration about what a large object is. In .net, a large object is considered to have 85,000 or more bytes. Do you really have such big objects or do you have a very large graph of smaller objects?

If it is a graph of smaller objects, then they are stored in the SOH (small object heap). In this case, if you are creating the objects and letting them go immediately, you will get the best benefit from the Garbage Collector's optimizations that assume a generational model. What I mean is that you either create objects and let them go to die or you keep them forever. Holding on to them just "for a while", or in other words, pooling, will just let them have promoted to higher generations (up to Gen 2) and that will kill the GC's performance, because cleaning up gen 2 objects is expensive (eternal objects in gen 2 are not expensive, however). Don't worry about memory fragmentation. Unless you're doing interop or fancy stuff like pinning objects, the GC is very effective in what comes to avoiding memory fragmentation - it compacts memory it frees from the ephemeral segment.

If you do indeed have very large objects (for instance, very big arrays), then it can pay to pool them. Notice however, that if the arrays contain references to smaller objects, pooling those will lead to the problems I talked about on the previous paragraph, so you should be careful to clean the array (have its references pointing to null) frequently (every iteration?).

Having that said, the fact that you're calling IDisposable is not cleaning objects. The GC does so. Dispose is responsible for cleaning unmanaged resources. Nevertheless, it is very important you keep on calling Dispose on every object whose class implements IDisposable (the best way is through finally) because you are potentially releasing unmanaged resources immediately and also because you are telling GC it doesn't need to call the object's finalizer, which would lead to the unnecessary promotion of the object, which as we saw, is a no no.

Bottom line, the GC is really good in allocating and cleaning up stuff. Trying to help it usually results in worse performance, unless you really know what is going on.

To really understand what I am talking about:

Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework

Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework 2

Large Object Heap Uncovered

Answer 2

Feels strange, but I'll try to answer it myself. May I'll get some comments on it:

Both platforms suffer from fragmentation if very large objects are allocated and freed in a heavy manner (loops). In case of unmanaged apps, allocations are made from the virtual address space directly. Usually the working array will be wrapped in a class (c++), providing operator overloads for nice short syntax, some reference handling and a destructor, which makes sure, the array is freed immediately when running out of scope. Nevertheless, the arrays requested do not have the same size all the time - if larger arrays are requested, the same address block cannot get reused which may lead to fragmentation over time. Furthermore, there is no way of finding the block, which exactly serves the array length requested. The OS would simply use the first block, which is large enough - even if it was larger as needed and may could have been better fullfilling the request for an later upcoming even larger array. How could pooling improve that situation?

Imaginable would be, to use larger arrays for smaller requests. The class would handle the transition from the true length of the underlying array to the virtual length needed for the outside world. The pool could help to deliver the "first array, which is long enough" - in contrast to the OS, which will always give the exact length. This could possibly limit fragmentation, since less holes are created in virtual address space. On the other side, the overall memory size would increase. For nearly random allocation patterns, pooling would bring little to no profit, but only eat rare memory, i guess.

On the managed side, the situation is worse. First of all, two possible targets for fragmentation exist: virtual address space and the managed large object heap. Latter in that case would more be a collection of individual seqments, individually allocated from the OS. Each seqment would mostly be used for a single array only (since we are talking of really big arrays here). If one array is freed by the GC, the whole segment is returned to the OS. So fragmentation would not be an issue in the LOH (references: my own thoughts and some empirical observations using VMMap, so any comments very welcome!).

But since the LOH segments are allocated from the virtual address space, fragmentation is an issue here too - just like for unmanaged applications. In fact, the allocation pattern for both applications should look very similar for the memory manager of the OS. (?) With one distinction: the arrays are freed by the GC all at the same time. However, "really large arrays" will produce a lot of pressure on the GC. Only a relatively small number of "really large arrays" can be hold at the same time until collection occurs. Basically, the application usually spends a reasonable amount of time (seen about 5..45%) in the GC, also because virtually all collections will be expensive Gen2 collections and almost every allocation will result in such a Gen 2 collection.

Here pooling may help considerably. Once the arrays are not freed to the OS but rather collected in the pool, they are immediately available for further requests. (This is one reason, why IDisposable is not only meant for unmanaged resources). The framework/library does only have to make sure, the arrays are placed in the pool early enough and to enable the reuse of larger arrays for situations, where actually a smaller size is needed.