3

I encounter this problem when running a genetic programming algorithm which uses eval.

To illustrate the problem, I narrowed it down to the following code fragment:

(loop []
  (do 
    (eval (list '+ (rand) (rand)))
    (recur)))

When I run the code the garbage collector unloads all the created $eval_n classes once from metaspace but on the second garbage collector invocation it hangs.

I use jdk1.8.0_102 with the follwing JVM options: -XX:MetaspaceSize=200m -XX:MaxMetaspaceSize=200m

After a while I get the following error:

CompilerException java.lang.OutOfMemoryError: Metaspace, compiling:(form-init2581690491924993906.clj:1:1)

EDIT: I added a screenshot of visualVM to show the behaviour, when the JVM hangs, the graph is not updated anymore, and it keeps using a full CPU core.

enter image description here

I also tried it using java 7 (without any JMV options), and I encounter the same problem with PermGen.

Any ideas how to avoid this problem?

EDIT:

The problem only occurs when I run it from a leinigen-REPL with eclipse-counterclockwise. If I run the code from a basic command line REPL the problem does not occur!

Dennis van den Berg
  • 81
  • 6
  • Just wondering, how do you know that "[it hangs] on the second garbage collector invocation"? And how do you run this code to get the error? I cannot reproduce it. Could you provide the exact commands? – user2609980 Sep 16 '16 at 21:56
  • @ErwinRooijakkers, I added a screenshot of the VisualVM graphs. – Dennis van den Berg Sep 17 '16 at 07:56
  • Thanks. I installed this program and ran your loop from Emacs with Cider in the REPL on Java 1.7.0_80. It runs fine and the garbage collection cycles are consistent. See http://i.imgur.com/wHYEmII.png. – user2609980 Sep 17 '16 at 09:23
  • @ErwinRooijakkers thanks for you effort. I wonder why you graph does not show any variance in the number of loaded classes. – Dennis van den Berg Sep 17 '16 at 09:44
  • In what way do you run the code? – user2609980 Sep 17 '16 at 09:47
  • the default is `MaxMetaspaceSize = 18446744073709547520 ` ... you're giving it less, any reason for that? – the8472 Sep 17 '16 at 10:50
  • @ErwinRooijakkers good question! I encounter the problem when ran it from eclipse with the counterclockwise plugin within a repl started from leiningen. I just tried running it directly from the command line the jvm does not hang! So it must be a problem with either leiningen or counterclockwise I suppose... – Dennis van den Berg Sep 17 '16 at 13:44
  • @the8472: If I give it the default, the problem shows up at a later time. I gave it less MaxMetaspaceSize to be able to reproduce the problem more quickly. – Dennis van den Berg Sep 17 '16 at 13:46
  • @DennisvandenBerg it is probably related to Counterclockwise then since I do not have the problem and also use Leiningen, but not Counterclockwise. – user2609980 Sep 17 '16 at 18:47

2 Answers2

0

I noticed steadily increasing memory consumption with the above example. Adding a System/gc

(loop [] (do (eval (list '+ (rand) (rand))) (System/gc)) (recur))

doubled CPU consumption but kept memory usage at what looked like a long term steady state.

Jonah Benton
  • 3,598
  • 1
  • 16
  • 27
  • Yes, but calling System/gc that frequently causes the program to be really slow, so I do not think this is a real solution. The VM should handle the gc itself. – Dennis van den Berg Sep 17 '16 at 07:43
0

I concluded this problem is not directly related to clojure eval or the JVM. It only occurs in combination with eclipse-counterclockwise and/or a leiningen-REPL.

Dennis van den Berg
  • 81
  • 6