Java 7 fails to collect permanent generation which is collected by java 5

Question

Does anybody know why java 7 fails to collect permanent generation of app, resulting in java.lang.OutOfMemoryError: PermGen, while java 5 collects the permanent generation and app runs well?

App does evaluation of jython expressions in the loop, one iteration is approx. 5 sec. Body of the loop looks like:

PythonInterpreter py = new PythonInterpreter();
py.set("AI", 1);
((PyInteger)py.eval(expr)).getValue()

Screenshots of jvisual vm taken for app running in java 7 and java 5.

In both cases the same parameters are used:

-Xmx700m 
-XX:MaxPermSize=100m
-XX:+HeapDumpOnOutOfMemoryError
-Xloggc:"C:\Temp\gc.log" -XX:+PrintGCDetails  -XX:-TraceClassUnloading -XX:+PrintClassHistogram

java 7 java 5

Did you check gc algorithms by logging into JConsole or some other tool? — kosa, Aug 14 '12 at 15:43
There might be different JVM startup settings. Is there something unusual in your application? like hot-loading code, custom class loaders etc? — Andy, Aug 14 '12 at 15:45
The GC people are always tweaking the GC, trying desparately to improve performance, so I'm not surprised there's such a difference. My __guess__ is that they're now focused on the 700Mb rather than the 100Mb, and are letting the permanent memory expand rather than expending the effort to control it. They're focused on high performance and staying within 700Mb rather than good numbers for PermGen. — RalphChapin, Aug 14 '12 at 16:34
you have heap dumps enabled. what does the heap dump tell you? perhaps there is something holding on to your memory in permgen such that it cannot be gc'ed. — jtahlborn, Aug 14 '12 at 16:37
If you want to analyse permgen usage in a heapdump this may be helpful: https://sites.google.com/site/eclipsebiz/The-Unknown-Generation-Perm — MilesHampson, Aug 15 '12 at 16:08

score 2 · Answer 1 · answered Aug 16 '12 at 08:08

Having a small example to reproduce the problem, I've found that program running in java 7 outside Eclipse does not suffer from memory leak in permanent generation.

import org.python.core.PySystemState;
import org.python.util.PythonInterpreter;

public class Test01 {

  public static void main(String[] args) throws Exception  {
    PySystemState.initialize();
    long startNanos = System.nanoTime();
    for(int i = 0; i < 450000; i++)    {
        PythonInterpreter pi = new PythonInterpreter();
        long elapsedNanos = System.nanoTime() - startNanos;
        int avgStepInMicros = (int)((elapsedNanos / 1000) / (i+1));
        final String code = String.format(
                "stepNo = %d + 1\n" +
                "if stepNo %% 100 == 0:\n" + 
                "  print 'stepNo: %%d,  elapsedMillis: %%d, avgStepInMicros: %%d' %% (stepNo, %d, %d)", i, elapsedNanos/1000000, avgStepInMicros);
        pi.exec(code);
    }
  }
}

MAT showed a debugger thread as garbage collector root.

GCRoot

Strange is that debugging app in java 5 does not have this problem.

Debugger holds onto those references? Damn. – MilesHampson Aug 17 '12 at 09:57 — MilesHampson, Aug 17 '12 at 09:57

score 0 · Answer 2 · answered Aug 15 '12 at 13:49

One possibility for the permgen leak is the Serializable interface implemented by each PyInteger being stored in a static class_to_type map (PyType.java:101), this is a Jython bug. The only interesting changes to permgen allocation between 5 and 7 that I am aware of are the removal of intern'd strings in 7 and some changes to direct byte buffer memory allocation, so instead the temporal behaviour of your graph might be explained by the unloading of types on each iteration in Java 5.

Java 7 fails to collect permanent generation which is collected by java 5

2 Answers2