How to provide exteranl jars at runtime while invoking the .class file

Question

I am trying to compile and run a java class at run time in eclipse which uses an external jar example : JSON library in this case.

I have successfully compiled the java code but when i am trying to invoke the method, it is giving me following error java.lang.reflect.InvocationTargetException

And it works fine when i add that required jar to the build path of the eclipse.I don't want to add the jar to the buildPath of the eclipse as it is a requirement to load the jars from an external path along with the build path provided in the eclipse.

Is there any way to add the external jar path while invoking the class which contains external jars at runtime in JAVACompiler?

Please help me in this issue.

This is the class which is to be compiled and run at runtime having external jars.

import org.json.JSONArray;
import org.json.JSONObject;

public class JSONPRINTERCLASS{
    public void printJson() {
        System.out.println("In the printJson method of JSONPRINTERCLASS class");

        String json = "[{\"Name\":\"Prakhar Agrawal\",\"Email\":\"155@abc.com\"},{\"Name\":\"Rahul Dhakad\",\"Email\":\"RD@qwerty.com\"}]";

        JSONArray array = new JSONArray(json);
        for(Object obj : array) {
            JSONObject jsonObj = (JSONObject)obj;
            System.out.println("jsonObj = "+jsonObj);
            System.out.println("============================================");
            System.out.println("Name = "+jsonObj.get("Name"));
            System.out.println("Email = "+jsonObj.get("Email"));

        }
    }
    public static void main(String as[]) {
        System.out.println("In the main method of JSONPRINTERCLASS class");
        new JSONPRINTERCLASS().printJson();
    }



}

This is my class which i am running to compile the JSONPRINTERCLASS

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Writer;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

public class StackInLineCompiler {

    public static void main(String[] args) {
        StringBuilder sb = new StringBuilder(64);

        StringBuilder stringbuff = new StringBuilder();
        try {
            InputStream is = new FileInputStream(
                    "/home/ist/Oxygen_workspace/NewProject/testcompile/JSONPRINTERCLASS.java");
            BufferedReader buf = new BufferedReader(new InputStreamReader(is));
            String line = buf.readLine();

            while (line != null) {
                stringbuff.append(line).append("\n");

                line = buf.readLine();
            }
        } catch (Exception e) {
            System.out.println();
            e.printStackTrace();
        }

        String fileAsString = stringbuff.toString();
        System.out.println("Contents : " + fileAsString);

        // Read more:
        // http://javarevisited.blogspot.com/2015/09/how-to-read-file-into-string-in-java-7.html#ixzz58OfOY4Rr

        File helloWorldJava = new File("/home/ist/Oxygen_workspace/NewProject/testcompile/JSONPRINTERCLASS.java");
        if (helloWorldJava.getParentFile().exists() || helloWorldJava.getParentFile().mkdirs()) {

            try {
                Writer writer = null;
                try {
                    writer = new FileWriter(helloWorldJava);
                    writer.write(sb.toString());
                    writer.flush();
                } finally {
                    try {
                        writer.close();
                    } catch (Exception e) {
                    }
                }

                /**
                 * Compilation Requirements
                 *********************************************************************************************/
                DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
                JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
                StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);

                // This sets up the class path that the compiler will use.
                // I've added the .jar file that contains the DoStuff interface within in it...
                List<String> optionList = new ArrayList<String>();
                optionList.add("-classpath");
                optionList.add(System.getProperty("java.class.path")
                        + ":/home/ist/Downloads/jar_To_compile/jar/json-20160810.jar");

                System.out.println("optionList = " + optionList);
                Iterable<? extends JavaFileObject> compilationUnit = fileManager
                        .getJavaFileObjectsFromFiles(Arrays.asList(helloWorldJava));
                JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, optionList, null,
                        compilationUnit);
                /*********************************************************************************************
                 * Compilation Requirements
                 **/
                if (task.call()) {
                    /**
                     * Load and execute
                     *************************************************************************************************/
                    System.out.println("Yipe");
                    try {
                        // Create a new custom class loader, pointing to the directory that contains the
                        // compiled
                        // classes, this should point to the top of the package structure!
                        URLClassLoader classLoader = new URLClassLoader(new URL[] { new File("./").toURI().toURL() });
                        // Load the class from the classloader by name....
                        Class<?> loadedClass = classLoader.loadClass("JSONPRINTERCLASS");
                        // Create a new instance...
                        Method declaredMethod = loadedClass.getDeclaredMethod("printJson");
                        // Santity check
                        System.out.println(
                                "Object Loaded Successfully...Now to call the method = " + declaredMethod.getName());
                        declaredMethod.invoke(loadedClass.newInstance(), null);
                        System.out.println("after invoking the method...........");
                    } catch (Exception e) {
                        System.out.println("In the exception while calling the method = " + e);
                    }

                    //
                    // }
                    /*************************************************************************************************
                     * Load and execute
                     **/
                } else {
                    System.out.println("In the error");
                    for (Diagnostic<? extends JavaFileObject> diagnostic : diagnostics.getDiagnostics()) {
                        System.out.format("Error on line %d in %s%n", diagnostic.getLineNumber(),
                                diagnostic.getSource().toUri());
                    }
                }
                fileManager.close();
            } catch (IOException exp) {
                exp.printStackTrace();
            }
        }
    }

}

Here at the last is the code after changes suggested by @AL4

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Writer;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import java.util.stream.Stream;

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

public class StackInLineCompiler {

    public static void main(String[] args) {
        StringBuilder sb = new StringBuilder(64);

        StringBuilder stringbuff = new StringBuilder();
        try {
            InputStream is = new FileInputStream(
                    "/home/ist/Oxygen_workspace/NewProject/testcompile/JSONPRINTERCLASS.java");
            BufferedReader buf = new BufferedReader(new InputStreamReader(is));
            String line = buf.readLine();

            while (line != null) {
                stringbuff.append(line).append("\n");

                line = buf.readLine();
            }
        } catch (Exception e) {
            System.out.println();
            e.printStackTrace();
        }

        String fileAsString = stringbuff.toString();
        System.out.println("Contents : " + fileAsString);

        // Read more:
        // http://javarevisited.blogspot.com/2015/09/how-to-read-file-into-string-in-java-7.html#ixzz58OfOY4Rr

        File helloWorldJava = new File("/home/ist/Oxygen_workspace/NewProject/testcompile/JSONPRINTERCLASS.java");
        File parentFile = helloWorldJava.getParentFile();

        final File jsonJarFile = new File("/home/ist/Downloads/jar_To_compile/jar/json-20160810.jar");

            try {


                /**
                 * Compilation Requirements
                 *********************************************************************************************/
                DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
                JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
                StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);

                // This sets up the class path that the compiler will use.
                // I've added the .jar file that contains the DoStuff interface within in it...
                List<String> optionList = new ArrayList<String>();

//              optionList.add("classpath");
//              
//              optionList.add(System.getProperty("classpath")
//                      + File.pathSeparator + jsonJarFile.getAbsolutePath());

                System.out.println("jar file path = "+jsonJarFile.getAbsolutePath());

                optionList.add("-classpath");
                optionList.add(System.getProperty("java.class.path")
                        + File.pathSeparator + jsonJarFile.getAbsolutePath());

                for (Diagnostic diagnostic : diagnostics.getDiagnostics()) {
                    System.out.println(diagnostic.getCode());
                    System.out.println(diagnostic.getKind());
                    System.out.println(diagnostic.getPosition());
                    System.out.println(diagnostic.getStartPosition());
                    System.out.println(diagnostic.getEndPosition());
                    System.out.println(diagnostic.getSource());
                    System.out.println(diagnostic.getMessage(null));
                }

                System.out.println("Now loading the jars at runtime");

                URLClassLoader classLoader = new URLClassLoader(Stream
                        .of(parentFile, jsonJarFile)
                        .filter(Objects::nonNull)
                        .map(StackInLineCompiler::toUrl)
                        .toArray(URL[]::new));



                ///////////////////////////////////////////////////////////////////////////////////////
//              
                System.out.println ("Success!");

//               
                 System.out.println("Jars loaded succesfully....");

                System.out.println("optionList = " + optionList);
                Iterable<? extends JavaFileObject> compilationUnit = fileManager
                        .getJavaFileObjectsFromFiles(Arrays.asList(helloWorldJava));
                JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, optionList, null,
                        compilationUnit);
                /*********************************************************************************************
                 * Compilation Requirements
                 **/
//              if (task.call()) {
                if (task.call()) {
                    /**
                     * Load and execute
                     *************************************************************************************************/
                    System.out.println("Yipe");
                    try {
                        // Create a new custom class loader, pointing to the directory that contains the
                        // compiled
                        // classes, this should point to the top of the package structure!
//                      URLClassLoader classLoader = new URLClassLoader(new URL[] { new File("./").toURI().toURL() });
                        Class<?> loadedClass = Class.forName("JSONPRINTERCLASS", true, classLoader);

                        Method declaredMethod = loadedClass.getDeclaredMethod("printJson");
                        // Santity check
                        System.out.println(
                                "Object Loaded Successfully...Now to call the method = " + declaredMethod.getName());


                        declaredMethod.invoke(loadedClass.newInstance(), null);
                        System.out.println("after invoking the method...........");
                    } catch (Exception e) {
                        System.out.println("In the exception while calling the method = " + e);
                    }

                    //
                    // }
                    /*************************************************************************************************
                     * Load and execute
                     **/
                } else {
                    System.out.println("In the error");
                    for (Diagnostic<? extends JavaFileObject> diagnostic : diagnostics.getDiagnostics()) {
                        System.out.format("Error on line %d in %s%n", diagnostic.getLineNumber(),
                                diagnostic.getSource().toUri());
                    }
                }
                fileManager.close();
            } catch (IOException exp) {
                exp.printStackTrace();
            }
//      }
    }

    // helper method
    static URL toUrl(File f) {
        try {
            return f.toURI().toURL();
        } catch (Exception e) {
            throw new RuntimeException(String.valueOf(f), e);
        }
    }

//  private static final Class<?>[] parameters = new Class[] { URL.class };

    /**
     * Adds a file to the classpath.
     * 
     * @param s
     *            a String pointing to the file
     * @throws IOException
     */
    /*public static void addFile(String s) throws IOException {
        File f = new File(s);
        addFile(f);
    }*/

    /**
     * Adds a file to the classpath
     * 
     * @param f
     *            the file to be added
     * @throws IOException
     */
    /*public static void addFile(File f) throws IOException {
        addURL(f.toURI().toURL());
    }*/

    /**
     * Adds the content pointed by the URL to the classpath.
     * 
     * @param u
     *            the URL pointing to the content to be added
     * @throws IOException
     */
    /*public static void addURL(URL u) throws IOException {
        URLClassLoader sysloader = (URLClassLoader) ClassLoader.getSystemClassLoader();
        Class<?> sysclass = URLClassLoader.class;
        try {
            Method[] methodarray = sysclass.getDeclaredMethods();
            for (Method method : methodarray) {

                System.out.println("Method name = " + method.getName());
                method.setAccessible(true);
            }

        } catch (Throwable t) {
            t.printStackTrace();
            throw new IOException("Error, could not add URL to system classloader");
        }
    }*/

}

But i am still getting

java.lang.Error: Unresolved compilation problems: JSONArray cannot be resolved to a type JSONArray cannot be resolved to a type JSONObject cannot be resolved to a type JSONObject cannot be resolved to a type at JSONPRINTERCLASS.printJson(JSONPRINTERCLASS.java:20) at JSONPRINTERCLASS.(JSONPRINTERCLASS.java:7) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at StackInLineCompiler.main(StackInLineCompiler.java:137)

Please help with this

A4L · Answer 1 · 2018-02-28T15:06:34.853

Your code has a couple of flaws:

It reads the content of the jar file which is absolutely not necessary
It does not check whether parentFile is null or not.
It deletes the content of the source file by writing an empty StringBuilder to it.
It uses : to setup the compiles class path which is not portable (will not work on windows).
It does not check the diagnostics object to see whether there were errors or not while compiling.
the URLClassload is not correct, is set the only current working directory in the runtime classpath, what you need is the location where the .class file of the compiled source (parentFile) and the location of the jar file (jar file itself)
It does not initialize the class loaded

How to Fix

get rid of this code

just get the parent file and save it for later, you will need it to setup the runtime classpath

File helloWorldJava = new File("java_sources/JSONPRINTERCLASS.java");
File parentFile = helloWorldJava.getParentFile();

get rid of this code

user File.pathSeparator to get the system specific path separator, for unix it is :, for windows it is ;

// define as constant
private final static File jsonJarFile = new File("3rdparty/json-20180130.jar");

setup your compile classpath like this:

optionList.add(System.getProperty("java.class.path")
            + File.pathSeparator + jsonJarFile.getAbsolutePath());

check the diagnostics object for compile issues

for (Diagnostic diagnostic : diagnostics.getDiagnostics()) {
    System.out.println(diagnostic.getCode());
    System.out.println(diagnostic.getKind());
    System.out.println(diagnostic.getPosition());
    System.out.println(diagnostic.getStartPosition());
    System.out.println(diagnostic.getEndPosition());
    System.out.println(diagnostic.getSource());
    System.out.println(diagnostic.getMessage(null));
}

setup the runtime classpath correctly

Is there any way to add the external jar path while invoking the class which contains external jars at runtime in JAVACompiler?

URLClassLoader classLoader = new URLClassLoader(Stream
    .of(parentFile, jsonJarFile)
    .filter(Objects::nonNull)
    .map(StackInLineCompiler::toUrl)
    .toArray(URL[]::new));

// helper method
public static URL toUrl(File f) {
    try {
        return f.toURI().toURL();
    } catch (Exception e) {
        throw new RuntimeException(String.valueOf(f), e);
    }
}

Load and initialize the class

Class<?> loadedClass = Class.forName("JSONPRINTERCLASS", true, classLoader);

After applying those steps you should be able to run the method printJson which should output:

In the printJson method of JSONPRINTERCLASS class
jsonObj = {"Email":"155@abc.com","Name":"Prakhar Agrawal"}
============================================
Name = Prakhar Agrawal
Email = 155@abc.com
jsonObj = {"Email":"RD@qwerty.com","Name":"Rahul Dhakad"}
============================================
Name = Rahul Dhakad
Email = RD@qwerty.com

Hello Thanks for the your help, but i am still getting compilation error while invoking the method, at the last in above question, i have pasted my new code with changes according to your suggestion.Please help — Prakhar Agrawal, Mar 01 '18 at 10:34
@PrakharAgrawal Are you sure that the file `jsonJarFile` really exists, the error message suggests that it is not present in the compile classpath, please add this debug line to see whether it exists or not: `System.out.printf("jsonJarFile exists? %b%n", jsonJarFile.exists());` — A4L, Mar 01 '18 at 10:58
The line ==> System.out.printf("jsonJarFile exists? %b%n", jsonJarFile.exists()); I have used it and it is showing true, means it is present there.I think thats the problem with the classpath, am i using this correctly as u suggested or not? optionList.add("-classpath"); optionList.add(System.getProperty("java.class.path") + File.pathSeparator + jsonJarFile.getAbsolutePath()); — Prakhar Agrawal, Mar 01 '18 at 11:54
@PrakharAgrawal I cannot reproduce the error you get with exact same code you posted after my answer, my first comment was misleading, `Unresolved compilation problem` means that you are trying to run a class which contains compilation errors. Try to add this line to delete all compiled class files (a.k.a clean) in the source location before compiling: `Stream.of(parentFile.listFiles(e -> e.getName().toLowerCase().endsWith(".class"))) .forEach(File::delete);` add this line after `File parentFile = helloWorldJava.getParentFile();`. — A4L, Mar 02 '18 at 07:36
@AL4 are you using eclipse or any other IDE for the running the code, if yes then please check whether you have added any json-jar in the build path of the IDE.Because if i also add that jar to the build path then it is running fine.But thats not the case i want. I have tried your code of removing class file and it didnt work. Please check on the above query. — Prakhar Agrawal, Mar 03 '18 at 09:35
@PrakharAgrawal Yes I am using eclipse, and no there is no json jar in the classpath, as test I remove the `jsonJarFile` from `optionList` and I get compile errors, what you get is runtime errors, there is a difference. Besides of that I did another test using only command line tools: `javac` for compiling and `java` for running, and it worked. So there is something in your setup or something you are doing that you don't tell. And again `java.lang.Error: Unresolved compilation problems` is a runtime error not a compile error. — A4L, Mar 03 '18 at 10:46

How to provide exteranl jars at runtime while invoking the .class file

1 Answers1