Avoiding PermGen out of memory and GC overhead limit exceeded

Question

I'm trying to generate classes and load them at run time.
I'm using a ClassLoader object to load the classes. Since I don't want to run out of PermGen memory, from time to time I un-reference the class loader and create a new one to load the new classes to be used. This seems to work fine and I don't get a PermGen out of memory. The problem is that when I do that, after a while I get the following error:

java.lang.OutOfMemoryError: GC overhead limit exceeded

So my question is, when should I un-reference the class loader to avoid both errors?:
Should I monitor in my code the PermGen usage so that I un-reference the class loader and call System.gc() when the PermGen usage is close to the limit?
Or should I follow a different approach?
Thanks

score 6 · Accepted Answer · answered Oct 24 '13 at 17:49

There is no single correct answer to this.

On the one hand, if unlinking the classloader is solving your permgen leakage problems, then you should continue to do that.

On the other hand, a "GC overhead limit exceeded" error means that your application is spending too much time garbage collection. In most circumstances, this means that your heap is too full. But that can mean one of two things:

The heap is too small for your application's requirements.
Your application has a memory leak.

You could assume that the problem is the former one and just increase the heap size. But if the real problem is the latter one, then increasing the heap size is just postponing the inevitable ... and the correct thing to do would be to find and fix the memory leak.

Don't call System.gc(). It won't help.

I'm using jvisualvm to monitor the heap usage, the maximum size is 1 Gb and the usage varies between 200 mb to 600 mb, so it is far from the limit. — pablo, Oct 24 '13 at 17:58
@otonakav - perhaps there is some problem with your GC tuning parameters. — Stephen C, Oct 24 '13 at 18:06

score 1 · Answer 2 · answered Oct 24 '13 at 17:54

1

Are you loading the same class multiple times? Because you should cache the loaded class.

If not, how many classes are you loading? If they are plenty you may have to fix a limit of loaded classes (this number can be either based on heap size or a number based on how much memory does it take to have a loaded class) and discard the least used when loading the next one.

answered Oct 24 '13 at 17:54

Bruno Costa

324
1
5
23

Probably a limit will do to minimize the number of times I un-reference a class loader so that the GC doesn't run too many times, but I was supposing the GC wouldn't run if there is plenty of heap and PermGen only because I un-reference a class loader, specially if it is going to pass the overhead limit. – pablo Oct 24 '13 at 19:04
I was trying to point out that the issue could be in the usage of the class loader and not the class loader itself. – Bruno Costa Oct 24 '13 at 19:19

Alexey Ragozin · Answer 3 · 2013-10-25T21:19:23.107

I had somewhat similar situation with class unloading.

I'm using several class loaders to simulate multiple JVM inside of JUnit test (this is usually used to work with Oracle Coherence cluster, but I was also successfully used this technique to start multi node HBase/Hadoop cluster inside of JVM).

For various reasons tests may require restart of such "virtual" JVM, which means forfeiting old ClassLoader and creating new one.

Sometimes JVM delays class unloading event if you for Full GC, which leads to various problems later.

One technique I found usefully for forcing JVM to collect PermSpace is following.

public static void forcePermSpaceGC(double factor) {
    if (PERM_SPACE_MBEAN == null) {
        // probably not a HotSpot JVM
        return;
    }
    else {
        double f = ((double)getPermSpaceUsage()) / getPermSpaceLimit();
        if (f > factor) {

            List<String> bloat = new ArrayList<String>();
            int spree = 0;
            int n = 0;
            while(spree < 5) {
                try {
                    byte[] b = new byte[1 << 20];
                    Arrays.fill(b, (byte)('A' + ++n));
                    bloat.add(new String(b).intern());
                    spree = 0;
                }
                catch(OutOfMemoryError e) {
                    ++spree;
                    System.gc();
                }
            }
            return;
        }
    }
}

Full sourcecode

I'm filling PermSpace with String using intern() until JVM would collect them.

But

I'm using that technique for testing
Various combination of hardware / JVM version may require different threshold, so it is often quicker to restart whole JVM instead of forcing it to properly collect all garbage

Avoiding PermGen out of memory and GC overhead limit exceeded

3 Answers3