6

While benchmarking a Clojure app and trying to pin down performance problems, I noticed this peculiar behavior: even when the entire program is written in Java, when launched from Leiningen it seems to experience a significant slowdown.

Say I have this Java program:

public class Foo {
    public static void main(String[] args) throws Exception {
        for (int i = 0; i < 10; i++)
             run();
    }

    public static void run() {
        final long start = System.nanoTime();

        Random r = new Random();
        double x = 0;
        for(int i=0; i<50000000; i++)
            x += r.nextDouble();

        final long time = TimeUnit.MILLISECONDS.convert(System.nanoTime() - start, TimeUnit.NANOSECONDS);
        System.out.println("time (ms): " + time + " total: " + x);
    }
}

When I simply run the program, I get execution times (per run) of about 1s. However, when I run it from leiningen like so:

lein run -m Foo

I get run times of about 2s! How does Clojure/Leiningen manage to slow down a complete Java program by so much? What am I doing wrong?

I've tried examining the system properties in both runs and couldn't find anything glaring (like different JIT settings). In both cases I use Java 7 with the server compiler.

EDIT: I don't know why this question has been downvoted. I'm not against Clojure. On the contrary, I love Clojure and I'm going to use it. I just have this serious performance problem that I absolutely must solve.

UPDATE: Running lein trampoline solves the issue! (though I have no idea why) I've updated the question to reflect that this is indeed a Leiningen issue, not a Clojure issue.

Another update: This happens for any Clojure code as well. Running without trampoline slows down code by up to 5x.

pron
  • 1,693
  • 18
  • 28

3 Answers3

3

The creators of leiningen are aware of this and give a thorough explanation of why this is so and what you can do about this.

https://github.com/technomancy/leiningen/wiki/Faster

Related question: Why is leiningen so slow when it starts?

Community
  • 1
  • 1
tixel
  • 69
  • 3
1

It's probably due to different JIT behaviour.

The performance of JIT compiled can be affected by a number of things, including:

  • What startup code gets called, which will affect JIT statistics
  • What other classes have been loaded (e.g. other subclasses of Random) which could affect the compiler's optimisation of method call dispatch
mikera
  • 105,238
  • 25
  • 256
  • 415
  • I've tried both with -XX:+PrintCompilation. In both cases run() is compiled (the only interesting function here). See the update, though. Somehow, it's leiningen causing this. – pron Apr 18 '13 at 12:19
-1

leiningen takes about a second to start.

Joost Diepenmaat
  • 17,633
  • 3
  • 44
  • 53
  • Obviously, I'm not counting lein's startup time. Look at the program. I'm looking at the timings in the run method. – pron Apr 18 '13 at 10:48
  • You're right. I'm guessing there is something weird going on there. Are you sure you've got the same things on the classpath? – Joost Diepenmaat Apr 18 '13 at 10:56
  • Also: you might have different memory limits, so the lein-started program may have to do more garbage collecting. The problem is hard to diagnose without the exact settings/system properties. – Joost Diepenmaat Apr 18 '13 at 11:02
  • No garbage is being collected at all while the program runs (verified with `-verbose:gc`). In order for garbage to be collected it has to be created first, and I don't think Clojure/lein can somehow make the algorithm produce more garbage. – pron Apr 18 '13 at 11:45