Is it possible to modify a ReadOnlyCollection using reflection

Question

I'm dealing with an SDK that keeps references to every object it creates, as long as the main connection object is in scope. Creating a new connection object periodically results in other resource issues, and is not an option.

To do what I need to do, I must iterate through thousands of these objects (almost 100,000), and while I certainly don't keep references to these objects, the object model in the SDK I'm using does. This chews through memory and is dangerously close to causing OutOfMemoryExceptions.

These objects are stored in nested ReadOnlyCollections, so what I'm trying now, is to use reflection to set some of these collections to null when I'm done with them, so the garbage collector can harvest the used memory.

foreach (Build build in builds)
{
        BinaryFileCollection numBinaries = build.GetBinaries();
        foreach (BinaryFile binary in numBinaries)
        {
            this.CoveredBlocks += binary.HitBlockCount;
            this.TotalBlocks += binary.BlockCount;
            this.CoveredArcs += binary.HitArcCount;
            this.TotalArcs += binary.ArcCount;

            if (binary.HitBlockCount > 0)
            {
                this.CoveredSourceFiles++;
            }

            this.TotalSourceFiles++;

            foreach (Class coverageClass in binary.GetClasses())
            {
                if (coverageClass.HitBlockCount > 0)
                {
                    this.CoveredClasses++;
                }

                this.TotalClasses++;

                foreach (Function function in coverageClass.GetFunctions())
                {
                    if (function.HitBlockCount > 0)
                    {
                        this.CoveredFunctions++;
                    }

                    this.TotalFunctions++;
                }
            }

            FieldInfo fi = typeof(BinaryFile).GetField("classes", BindingFlags.NonPublic | BindingFlags.Instance);
            fi.SetValue(binary, null);
    }

When I check the values of the classes member in numBinaries[0], it returns null, which seems like mission accomplished, but when I run this code the memory consumption just keeps going up and up, just as fast as when I don't set classes to null at all.

What I'm trying to figure out is whether there's something intrinsically flawed in this approach, or if there's another object keeping references to the classes ReadOnlyCollection that I'm missing.

have you tried reflector to see what your API is really doing. A read-only collection is just a reference to a read/write collection. So unless you can get to the source, it won't work (probably). I am assuming you have tried your SDK vendor — Adam Straughan, Mar 08 '11 at 20:00
I've been looking through the SDK using reflector, which is how I know about the private classes field. I would think that if I make the object that contains the reference to the read only collection null, then the GC will see that the read only collection has no references to it, and that its internal list has no references and clean that space on the heap up. — user467384, Mar 08 '11 at 20:03
_"what I'm trying now, is to use reflection to set some of these collections to null when I'm done with them"_ -- almost certainly the wrong/worst way to approach this problem. That said, even taken literally, the question is too broad as it depends entirely on how the collections are used, and even what collection classes they are exactly. And really, this should be a different question, specifically about how to deal with the library in question and its excessive memory requirements. Mucking around readonly collections using reflection is guaranteed trouble — Peter Duniho, Jan 20 '20 at 23:55

vcsjones · Answer 1 · 2020-01-20T22:14:24.250

I can think of a few alternatives...

Logically split it out. You mentioned it keeps all the references "for the duration of a connection". Can you do 10%, close it, open a new one, skip that 10%, take another 10% (total 20%), etc?
How much memory are we talking about here, and is this tool going to be something that is long-lived? So what if it uses a lot of RAM for a few minutes? Are you actually getting OOMs? If your system has that much available RAM for the program to use, why not use it? You paid for the RAM. This reminds me of one of Raymond Chen's blog posts about 100% CPU consumption.
If you really want to see what is keeping something from getting garbage collected, firing up SOS and using !gcroot is a place to start.

But despite all of that, if this really is a problem, I would spend more time with the 3rd party API provider - at some point they may release an update you want that breaks this - and you'll be back to square one, or worse you can introduce subtle bugs in the product.

1. I tried logically splitting it out (that was the first thing I tried), but the root object has a lot of references, and ends up throwing a TimeOutException because it runs out of connections in its connection pool. 2. 300MB for a single build. I run into OOMs after about 5-6 builds, and since we're a new team, it's not impossible that our number of functions will increase 5-6x. I actually talked with the SDK team, and they know about this issue, and invited me to fix it in their source code :). — user467384, Mar 08 '11 at 22:13

Is it possible to modify a ReadOnlyCollection using reflection

1 Answers1