2

For benchmarking a multithreaded algorithm, I set up a parameterized jmh measurement. After quite some time, the measurement crashes with the error

java.lang.OutOfMemoryError: unable to create new native thread

The algorithm creates a very limited number of threads and does not leak memory. This question is therefore different from the many other questions here on SO from people who got that error within a single application.

The problem apparently stems from the many executions that jmh performs over a long period of time, where apparently, the OS (Ubuntu 14.04, 64-bit) or the JVM doesn't properly clean up native threads allocated in previous executions.

The benchmark is started from a script (inside Eclipse, not using the command line) where the Options object is built the following way:

Options opts = new OptionsBuilder()
            .include(".*")
            .warmupIterations(5)
            .measurementIterations(5)
            .jvmArgs("-server", "-Xmx2G")
            .forks(1)
            .resultFormat(ResultFormatType.CSV)
            .output(file.getPath())
            .shouldFailOnError(true)
            .shouldDoGC(true)
            .build();

The question then is: How are concurrent algorithms properly benchmarked? Should the measurement be performed in a specific way? Specific parameters set on jmh/JVM? Should I increase the allowed number of native threads as suggested in other answers or will that affect the result?

Since nanosecond precision is not strictly needed, I don't necessarily need jmh, but I like its interface, precision, and features. Also, I suspect that the problem is more related to OS/JVM than to jmh itself.

bisgardo
  • 4,130
  • 4
  • 29
  • 38

1 Answers1

0

It turns out that there indeed was a memory leak, where some threads sometimes would wait forever after having written their results to shared memory.

bisgardo
  • 4,130
  • 4
  • 29
  • 38
  • 1
    Please explain? Those were user threads, i.e. started by your own code? Those threads should have died when forked VM exited, why wouldn't they? – Aleksey Shipilev May 19 '15 at 06:38
  • I think I ended up creating more threads over the trial of measurements than I originally thought. Anyway, I have not been able to create a minimal example of the problem, and edited the answer accordingly. – bisgardo May 21 '15 at 00:57