[NOTE: From November 2015 it is no longer open source, so this hack is mostly useless.]
RoboVM might become the solution you're looking for. It's open source and compiles JVM bytecode (.class files) to machine code.
I assume they do it using something like class2llvm.
Unfortunately, it's still in alpha. I just tested it on HelloWorld.java. It gave 5x speed up of load time running on a single core. (Most of the run time is load time.)
echo Hello World! : <1 ms : 31K
(/usr/bin/echo binary)
java HelloWorld : ~70 ms : 0.4K
(HelloWorld.class JVM bytecode)
./HelloWorld : ~13 ms : 9.4MB
(9.3MB binary + 57K robovm-rt.jar)
Note that java calls a 32MB $JAVA_HOME/lib/rt.jar file (and maybe more). Searching in such a large file must be part of the reason java is so slow to load. If RoboVM gets smarter, perhaps it can throw out most of the 9.3MB binary for an even faster load?
The website mentions iOS, but I think that's because they're selling their add-on UI libraries. RoboVM compiled fine for me on a flavor of Ubuntu. Just make sure to do
$ sudo apt-get install g++-multilib
first (and maybe install libpthread-stubs0-dev and libpthread-workqueue0...don't know if they mattered).