0

I have a situation where I'm allocating a large chunk of memory, but in many smaller sub-arrays. I noticed the performance is significantly worse once I pass a threshold of around 85,000 bytes per array. I'm assuming the performance drop is because the smaller arrays are being allocated on the "small object heap" (SOH) rather than the "large object heap" (LOH) at that point.

# of Arrays Size of Each Array (bytes) Allocation Time
1,154 532,480 377ms
1,319 465,920 412ms
1,539 339,360 439ms
1,847 332,800 435ms
2,308 266,240 446ms
3,077 199,680 491ms
4,616 133,120 514ms
9,231 66,560 4420ms

Note, in all cases, the total memory allocated is around 586MB, but the allocation takes an order-of-magnitude longer in the final case.

My first thought to quickly resolve this performance issue was to somehow tell the C# runtime that I want those arrays in the large object heap even though they're smaller than the threshold. I assume this would bring the allocation time in line with the rest of the cases.

However, I can't seem to find if there's a way to do this. It looks like no one has ever wanted an object to go on the large object heap before. So, I ask: is it possible to flag these arrays in some way to force them onto the large object heap, even though they're smaller than the 85,000 byte threshold?

(A way to lower the 85,000 byte threshold to, say, 65,000 bytes, would also solve my problem, but I couldn't find a way to do that either!)

gfrung4
  • 1,658
  • 3
  • 14
  • 22
  • This is like running away from the circus to join the orphanage (aka opposite land). There are many solves to allocations problem, GC Pressures, and collection times. Non of them usually involve shooting the hostage you a trying to save. – TheGeneral Nov 18 '21 at 21:54
  • Did you ever try to allocate 9231 larger arrays? It may be that it's not the size but the object count that triggers something (e.g. a garbage collection or other housekeeping). – Peter - Reinstate Monica Nov 18 '21 at 21:54
  • @Peter-ReinstateMonica Yes, allocating 9231 arrays of length 85000 takes ~670ms. – gfrung4 Nov 18 '21 at 22:01
  • 2
    I have never seen a setting to reduce the LOH cut off limit. However, id be more concerned about this being an X/Y problem. If these are short lived arrays, have you considered the ArrayPool ? If you did go down the LOH path, sooner or later you will likely run into more problems like memory fragmentation etc – TheGeneral Nov 18 '21 at 22:02
  • Can you please show how did you measure the allocation time? (And the allocation code). I would say in case of not very well written test you can end up measuring not just allocation but GC for SOH case. – Guru Stron Nov 18 '21 at 22:10
  • That is not possible. Beware that a synthetic test is quite dangerous, the SOH allocation time includes the cost of running ~75 garbage collections. Background GC can't do its job in such a test since you'd run out of the gen #0 and #1 segments too quickly. That cost exist for LOH allocations as well, but avoids being measured when you only profile the allocation cost. – Hans Passant Nov 18 '21 at 22:11
  • As far as I know,, allocating an array costs _very little_; it's not much more than some support code around an `InterlockedIncrement`. It should be O(1) relative to the size of the array (every array allocated will do the same thing, it's just that the amount allocated changes). But, allocating that much memory likely forces the GC into action, and that's what's dragging you down. Get PerfMon and the .NET memory counters to look at what's going on. An array pool is often the cure for LIH problems – Flydog57 Nov 18 '21 at 22:26
  • An obvious solution if you have enough memory is to increase object size with dummy space. – Peter - Reinstate Monica Nov 18 '21 at 22:27
  • @GuruStron @Hans Passant The array's should not be getting garbage collected, as I'm keeping a reference to each one. They're getting added to a `List`, resulting in a list of all the arrays at the end. This allocation was actually in code running on an embedded system, and the performance hit was noticed, so it was not a synthetic test. All I did was add some `Stopwatch` lines around the part where arrays are allocated and noted the times, all running on the real-world system. – gfrung4 Nov 18 '21 at 22:28
  • @Peter-ReinstateMonica Ha! I only need to support the eight cases in the table above, so bumping up the last one to 85,000 byte arrays was solution (b), since that seems to work at the cost of wasting some memory. – gfrung4 Nov 18 '21 at 22:29
  • @gfrung4 note that despite the reference being present GC still can kick due to Gen 0/ Gen 1 being full and the arrays being promoted to next generation. While it should be quite quick it will add some overhead. – Guru Stron Nov 18 '21 at 22:40
  • Also take a look at [`GC.AllocateArray`](https://learn.microsoft.com/en-us/dotnet/api/system.gc.allocatearray?view=net-6.0) and [`GC.TryStartNoGCRegion`](https://learn.microsoft.com/en-us/dotnet/api/system.gc.trystartnogcregion?view=net-6.0), maybe it can help. – Guru Stron Nov 18 '21 at 22:42
  • @TheGeneral Thanks for letting me know about `ArrayPool`! It looks like exactly what I want, with one exception: it appears to have a limit of 1GB max size. This limitation is already why I wasn't using one big array... While I'm currently only allocating ~586MB of memory, this application runs inside an embedded device that can have up to 4GB of memory, and it will need to use nearly *all* that memory to capture a significant amount of data. I think I'm just running into a limitation of C# as a language here, and should probably consider C or C++ for such a memory-intensive task. – gfrung4 Nov 18 '21 at 22:46
  • Can you explain the actual use case for the arrays. Do you create them once? or do you tend to add and remove them from the list? Are all the arrays needed at once? how many arrays are actually needed at once. Answers to these sorts of questions might be able to give you a better solution – TheGeneral Nov 18 '21 at 22:50
  • There are also many other combinations of solutions depending on the use cases and environment, I.e you could have just one large memory mapped file and create a memory manager around the to access it via Memory and spans ect – TheGeneral Nov 18 '21 at 22:53

1 Answers1

0

After reading https://learn.microsoft.com/en-us/dotnet/standard/garbage-collection/large-object-heap, I assume that the small objects, starting in heap generation 0, are subject to more frequent garbage collections. Those that are not collected because they are still referenced may be moved for compacting.

Objects on the large object heap are kinda-treated like generation 2 objects and less frequently collected to begin with. And when they are, the remaining ones are not moved because it's considered too expensive. (After reading in your comment that you already keep a reference to each array, the issue with small objects is probably this moving for compaction.)

Solution: Prevent garbage collection of your small objects by keeping references to them, and pin them in order to prevent moves for compacting when other objects are collected.

(Whether essentially disabling garbage collection is doable or advisable depends on the circumstances. You are essentially on your own with memory management and must, when the occasion arises, unpin (and, if due, un-reference) your objects to make them available for collection or compaction, or you'll run into memory or fragmentation problems.)

Peter - Reinstate Monica
  • 15,048
  • 4
  • 37
  • 62