How to safely access the URLs of all resource files in the classpath in Java 9+?

Question

We learned from the release notes of Java 9 that

The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.

This breaks old code, which scans the classpath as follows:

Java <= 8

URL[] ressources = ((URLClassLoader) classLoader).getURLs();

which runs into a

java.lang.ClassCastException: 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to 
java.base/java.net.URLClassLoader

So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.

/*
 * Java 9 + Bridge to obtain URLs from classpath...
 */
private static URL[] getURLs(ClassLoader classLoader) {
    URL[] urls = new URL[0];

    try {
        //see https://github.com/apache/ignite/pull/2970
        Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");

        if (builtinClazzLoader != null) {
            Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
            ucpField.setAccessible(true);

            Object ucpObject = ucpField.get(classLoader);
            Class clazz = Class.forName("jdk.internal.loader.URLClassPath");

            if (clazz != null && ucpObject != null) {
                Method getURLs = clazz.getMethod("getURLs");

                if (getURLs != null) {
                    urls = (URL[]) getURLs.invoke(ucpObject);
                }
            }
        }

    } catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
        logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
        logger.error(e.getLocalizedMessage(), e);
    }
    return urls;
}

However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:

Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]

Question

Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?

UPDATE (related to the comments)

I know, that I can do

 String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));

to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.

UPDATE 2

Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.

However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.

UPDATE 3

Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.

If you want the URLs to the elements of the class path then look at the system property java.class.path. — Alan Bateman, Mar 29 '18 at 15:14
Yes - thanks for your comment. But we need this information for the given classloader (e.g. in an application server). Question was a bit unspecific, so I updated it. — rzo1, Mar 29 '18 at 16:18
It's easy to split the value of the java.class.path property and create a file URL to each element, that is exactly what the application class loader does when it is initialized. — Alan Bateman, Mar 29 '18 at 16:20
`System.getProperty("java.class.path")` still works and you can make URLs of it with ease. But the key point is, that’s the class path. The module path might be more involved and nobody says, that it must be representable as a list of `URL`s at all. I suppose, that’s one of the reasons for moving away from the `URLClassLoader` as application loader; you are not supposed to assume that you are running on a bunch of `URL`s. — Holger, Mar 29 '18 at 17:00
“*this property only returns the classpath given at the time of the application launch*”—exactly and since Java 9 is going to forbid hacking the class path, that’s the only class path for the application class loader. But when you are talking about a container, you are not talking about the application loader at all. As long as you haven’t changed your application server, the container loader likely still is a`URLClassLoader`s. In the end, it seems to be an [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem/66378#66378). What do you want to do with those `URL`s? — Holger, Apr 05 '18 at 14:14
What you had in mind when you mention `e.g. by using Unsafe` ? — sujit, Apr 06 '18 at 06:19
By the way, the PR you mention was actually not merged but the team has since followed a similar approach as yours in their [IgniteUtils.java#classLoaderUrls()](https://github.com/apache/ignite/blob/a06470212ff1a736b59c4649e19c204fa1e74017/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L7589), although with a bug that I have now raised [here](https://issues.apache.org/jira/browse/IGNITE-8146) — sujit, Apr 06 '18 at 06:31
@sujit You could still use `Unsafe.objectFieldOffset(Field)` and `Unsafe.getObject(Object, long)` to access the `ucp` (in BuiltinClassLoader) and `path` (in URLClassPath) members. That should work without the need for `--add-opens`. — Stefan Zobel, Jun 18 '18 at 16:51

score 18 · Answer 1 · answered Apr 09 '18 at 10:51

I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).

Please could you expand on why you want to do this?

If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:

A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line. For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs

Thanks for your answer. I will taking a look into your suggestions. — rzo1, Apr 09 '18 at 10:57
+1 There is simply no *safe* way to get all classpath sources as URLs. Since Java 1.0, the Java VM has been able to load class files from arbitrary sources, not restricted to whatever is 'reachable' through an URL. The classloader is not required to offer more functionality than to be able to load a class when provided the class name as a string. Classpath browsing and access to internal URLs will work with some classloaders, but is simply not available in other runtime environments. — jarnbjo, Apr 19 '18 at 13:45
But for people who have to write code that is Java8-backward compatible, the new APIs, such as layers and services, are no help. Jorn Vernee's snippet and a new URL-encoder that wraps the class's loader sticks to java8 APIs and that is helpful in its own right. — Canonical Chris, Sep 19 '18 at 07:11
@CanonicalChris: the ServiceLoader API was added in 1.6 and is not new — Rich, Sep 19 '18 at 09:07

score 10 · Answer 2 · answered Mar 29 '18 at 13:52

10

AFAIK you can parse the java.class.path system property to get the urls:

String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
    result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}

System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]

answered Mar 29 '18 at 13:52

Jorn Vernee

31,735
4
76
93

1

This dependes on the environment, I guess. Imagine for example a TomEE application server with different classpath loader views. At the moment, our production code uses this approach in addition to the code sample from above to obtain a "more" complete view of the URLs in the execution environment / context. – rzo1 Mar 29 '18 at 13:56
1

@rzo I'm not familiar. You mean where a separate class loader is used? In that case, I'd assume it would be a URLClassloader again, and could expose api to get the urls. Any ways, if this doesn't solve the problem, you might want to add the extra specifics about your environment to your question. – Jorn Vernee Mar 29 '18 at 14:02
1

There are many environments in which this will not work, including EE containers like Tomcat, IDEs like IntelliJ, custom classloaders like Spring Boot, etc. etc. Please see my answer below. – Rich Apr 09 '18 at 10:55

How to safely access the URLs of all resource files in the classpath in Java 9+?

Question

2 Answers2

Linked

Related