16

I was wondering why the JVM's JIT compiler ignores "huge methods" from compilation. (Unless the DontCompileHugeMethods flag is set to false.) At the same time, most talks about Java's JIT compiler state that inlining is an uber-optimization as it allows to grow the mass of instructions that are subject to compilation. And this larger compilation context allows for a better optimization of the executed code. With this, I would assume that a huge method is not much different to a heavily inlined method and should be a great target for JIT compilation. What am I missing here?

Rafael Winterhalter
  • 42,759
  • 13
  • 108
  • 192
  • 1
    Related article: http://blog.leenarts.net/2010/05/26/dontcompilehugemethods/ – user2864740 Jul 24 '14 at 07:44
  • Related question: http://stackoverflow.com/questions/15756075/is-it-true-that-having-lots-of-small-methods-helps-the-jit-compiler-optimize?rq=1 – Thilo Jul 24 '14 at 07:47
  • Thanks for the related Q&As. I am however mostly wondering about the rationale. I know about what happens and I even investigated it using JIT watch. It is even possible to "manually inline" a deep call stack into one method where the same code sequence is JIT compiled without the "manual inlining" but ignored with the inlining. – Rafael Winterhalter Jul 24 '14 at 07:59
  • Simply that the longer the method, the less the benefit from compiling it, and this is hyperbolic in nature, not linear. – user207421 Jul 24 '14 at 09:33
  • What contradicts the rationale of inlining where, the longer the method, the better it can be optimized. – Rafael Winterhalter Jul 24 '14 at 09:46
  • @raphw No. It is the exact opposite. The longer the method, the *less* the benefit of inlining, which only amounts to saving the call and return. You're now confusing inlining with *compiling.* They're not the same thing. – user207421 Jul 24 '14 at 10:57
  • @EJB I heard several talks of Oracle JIT people where they state that virtual calls are not itself expensive but the fact that virtual calls do not allow for inlining where the methods do not get big enough to apply proper optimizations. Inlining is used to make methods "bigger" in order to optimize them in the course of JIT compiling. – Rafael Winterhalter Jul 24 '14 at 11:39

2 Answers2

10

Basically the ROI of compiling huge methods is low.

  • Hot pieces of code are usually short.

  • Even if a huge method is executed frequently, the hot part is unlikely to cover the whole method. E.g. consider a large switch statement where only a few case labels are executed often. However the compilation unit is a method - so the bigger part of JITted code would be a waste.

  • Compilation of huge methods takes much time and space. Moreover, the compilation time does not grow linearly. It would be more profitable to compile several small methods instead.

  • Too long sheet of machine code pollutes instruction cache.

  • Certain optimizations are better applied to smaller pieces of code, e.g. register allocation or loop unrolling.

  • If a single method is larger than 8K bytecodes, it seems to be poorly written anyway.

apangin
  • 92,924
  • 10
  • 193
  • 247
  • 1
    I doubt many methods would exceed 8kb when written. But does the JIT itself grow them by inlining methods and unrolling loops? – Raedwald Aug 05 '14 at 23:42
  • 1
    @Raedwald JIT optimizations are performed on IR, not on the bytecode level. Though JIT takes into account the sizes of the caller and the callee when deciding whether to inline method or not. – apangin Aug 06 '14 at 12:01
1

Th 8k limit probably just an outdated heuristic. Back in the days [TM], JITing was somewhat costly (especially when viewed from a responsiveness point-of-view). In fact, most interesting optimizations (think constant-folding, good register-allocation etc.) are super-linear. Hence you want to be extra careful not to stop the whole process for half a second to run an optimization task that might only yield a fraction of that time as performance gain.

Nevertheless, I guess with the grown experience, faster processors and better JIT-techniques, the limit could be somewhat lifted (probably even by an order of magnitude), but the core problem of super-linear performance remains and so will the limit.

choeger
  • 3,562
  • 20
  • 33
  • It should not be so outdated; even with better hardware it is no more sensible to write methods as big as that. There is always a way to refactor that to a more readable (and JITable) code... – Uwe Allner Jul 24 '14 at 07:51
  • 1
    But is JIT compilation not always an investment? It is only applied to hot methods anyways. So even if you spend a second on performing a JIT, it might pay off if the hot method is executed a lot. Also, JIT runs on its own thread, so latency should not be too big of a problem. – Rafael Winterhalter Jul 24 '14 at 08:05
  • JIT might nowadays run in parallel, but when the Hotspot VM was bought by Sun, Intel started producing the Pentium II ;). And yes, you might gain something even if JITing takes a long time. That's why I called it a heuristical approach. – choeger Jul 24 '14 at 08:11
  • But then, why would I decide to simply name a specific amount of bytes where JIT is not longer worth it. It just does not make sense to me. – Rafael Winterhalter Jul 24 '14 at 10:38
  • Because it is the measure for the input size of several optimization algorithms. So that's where you put your limit. – choeger Jul 24 '14 at 12:05