Java runtime compilation and class file generation

Question

Working on an application which requires several source files(.java) to be compiled and corresponding class files(.class) to be generated at runtime and this happens at application start-up.

Currently using below code for class generation:

int errorCode = com.sun.tools.javac.Main.compile(new String[] {
        "-cp", classPath,
        "-d", pOutputPath,
        srcFile.getAbsolutePath() },new PrintWriter(out));

Every *.java file is hardly some 40 lines of code with a single method. But the time the above given code takes to compile is around 2 seconds per file.

Application has more than 1000 or sometimes 2000 java files. So application start up time is more than 2000 or 4000 seconds which is undesirable.

Any alternative for com.sun.tools.javac.Main.compile?

Or a better or faster way for runtime compilation and class file generation?

I can't use multi-threading as the environment is single-threaded.

Can't you use a `CachedThreadPool` and fire jobs to compile files? — TheLostMind, Nov 06 '15 at 06:04
Actually a single thread is iterating over files in directory and compiling. If I have multiple threads also then I need to have lot of thread-safety measures like locks, sync, etc.. Searching for an alternative to Main.compile which is faster.. — Xavier DSouza, Nov 06 '15 at 06:07

score 1 · Answer 1 · answered Nov 06 '15 at 07:15

There is no need to compile your source files one-at-a-time. You can compile all of them in one go (just put them all in your String[]).

That will be much faster, as most of the two seconds is spent on initializing things, and only a fraction is spent on compiling a source file.

Say - as an estimate - that 0.2 seconds is spent on compiling the source file and 1.8 second on initializing things. Then compiling 2000 files would take 1.8 + 2000 * 0.2 = 401.8 seconds. That's just a guess; it may vary when you try it for real.

And you may also want to check whether the source files have actually changed since the last time that you compiled them, and only compile then again when at least one of them has changed.

It's much easier to justify waiting - given the above guess - 7 minutes to start up when there is an actual change in the code than when there is not.

No compiler takes 0.2 seconds to compile a single source file. I've written a compiler for a language that has significantly more features than Java, and it takes .3 secs for initialization and around 10 ms per source file. — Clashsoft, Nov 07 '15 at 13:09
@Clashsoft yes and I've written a java compiler. I'm putting forth a suggestion here how to calculate this; as the OP says compiling 1 source file takes 2 seconds, your example which takes .31 seconds to compile one source file is clearly not applicable. And of course duration depends on source file length/complexity, computer speed, etc. — Erwin Bolwidt, Nov 07 '15 at 14:13

pcejrowski · Answer 2 · 2015-11-07T12:54:34.597

Recently, I used ToolProvider in similar case: javax.tools.ToolProvider.getSystemJavaCompiler()

Example:

protected static boolean compileFiles(List<String> additionalOptions,ArrayList<String> sources){
  JavaCompiler compiler=ToolProvider.getSystemJavaCompiler();
  if (compiler == null)   return false;
  StandardJavaFileManager fileManager=compiler.getStandardFileManager(null,null,null);
  List<String> options=new ArrayList<String>();
  options.add("-d");
  options.add(getOutBuildDir().getAbsolutePath());
  options.add("-s");
  options.add(getOutSrcDir().getAbsolutePath());
  options.add("-verbose");
  if (additionalOptions != null)   options.addAll(additionalOptions);
  Iterable<? extends JavaFileObject> compilationUnits=fileManager.getJavaFileObjectsFromStrings(sources);
  CompilationTask task=compiler.getTask(mOutputWriter,fileManager,null,options,null,compilationUnits);
  return task.call();
}

from here: http://www.programcreek.com/java-api-examples/javax.tools.ToolProvider

Another way to compile in runtime is in memory compilation using e.g. JIMCy library(https://github.com/Krever/JIMCy). It is actually written in Scala, but has Java API.

That's certainly a nicer way to invoke the Java compiler (but doesn't make it faster unless you also do other things). However at the moment it's a link-only answer and links go stale after a while. Please include the relevant information and write it in your own answer. — Erwin Bolwidt, Nov 06 '15 at 07:28

Dima Maligin · Answer 3 · 2015-11-06T07:05:29.513

0

Why not let the OS handle it for you?

I'm not to sure about this but I assume you can try something like:

String path = "srcFolderPath";
File dir = new File(path);
for (File src : dir.listFiles()) {

    try {
        ProcessBuilder pb = new ProcessBuilder()
                .directory(dir.getAbsoluteFile())
                .command("javac " + src.getName()); // or whatever command params

        Process p = pb.start();
    } catch (IOException e) {
        e.printStackTrace();
    }

}

PS: I haven't tested it but you can try.

edited Nov 06 '15 at 07:05

answered Nov 06 '15 at 06:35

Dima Maligin

1,386
2
15
30

Why do you think that this is faster? – Erwin Bolwidt Nov 06 '15 at 07:26

Java runtime compilation and class file generation

3 Answers3