Maven jlink plugin: native libraries not found when running the resulting image

Question

Context: Linux Mint 21.1, Java Adoptium 17.0.7 JDK, Maven 3.6.3.

My modular application uses LWJGL (and then its native libraries); using the Maven jlink plug-in I generate a Java image. During generation, the jlink plug-in prints the following:

[INFO] --- maven-jlink-plugin:3.1.0:jlink (default-cli) @ treni ---
INFO]  -> module: org.lwjgl.stb ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-stb/3.3.2/lwjgl-stb-3.3.2.jar )
[INFO]  -> module: org.slf4j ( /home/mmg/.m2/repository/org/slf4j/slf4j-api/2.0.7/slf4j-api-2.0.7.jar )
[INFO]  -> module: imgui.natives.linux ( /home/mmg/.m2/repository/io/github/spair/imgui-java-natives-linux/1.86.10/imgui-java-natives-linux-1.86.10.jar )
[INFO]  -> module: org.lwjgl ( /home/mmg/.m2/repository/org/lwjgl/lwjgl/3.3.2/lwjgl-3.3.2.jar )
[INFO]  -> module: imgui.binding ( /home/mmg/.m2/repository/io/github/spair/imgui-java-binding/1.86.10/imgui-java-binding-1.86.10.jar )
[INFO]  -> module: org.lwjgl.stb.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-stb/3.3.2/lwjgl-stb-3.3.2-natives-linux.jar )
[INFO]  -> module: org.lwjgl.glfw ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.2/lwjgl-glfw-3.3.2.jar )
[INFO]  -> module: org.joml ( /home/mmg/.m2/repository/org/joml/joml/1.10.5/joml-1.10.5.jar )
[INFO]  -> module: org.lwjgl.opengl ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-opengl/3.3.2/lwjgl-opengl-3.3.2.jar )
[INFO]  -> module: org.slf4j.jul ( /home/mmg/.m2/repository/org/slf4j/slf4j-jdk14/2.0.7/slf4j-jdk14-2.0.7.jar )
[INFO]  -> module: org.lwjgl.opengl.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-opengl/3.3.2/lwjgl-opengl-3.3.2-natives-linux.jar )
[INFO]  -> module: org.lwjgl.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl/3.3.2/lwjgl-3.3.2-natives-linux.jar )
[INFO]  -> module: com.vistamaresoft.treni ( /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/target/classes )
[INFO]  -> module: org.lwjgl.glfw.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.2/lwjgl-glfw-3.3.2-natives-linux.jar )
[INFO] Building zip: /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/target/treni-0.0.1.zip

which shows that the plug-in knows about all the required native library JAR's.

I expected the resulting application to be able to locate the (evidently included, see below) native libraries, but it seems it is not, at leat without additional hits I have no idea how to give it (and I could not find described anywhere). In fact, when I run the generate shell script, I receive the error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Failed to locate library: liblwjgl.so

Note that the required library is contained in the lwjgl-3.3.2-natives-linux.jar referenced by the pom.xml dependencies via the appropriate profile (see below for the pom.xml contents).

The shell script is the default one generated by the jlink plug-in (the same for all my jlink-ed apps, only the module/main_class changes from one to another); anyway for completeness, this is its contents:

#!/bin/sh
JLINK_VM_OPTIONS=
DIR=`dirname $0`
$DIR/java $JLINK_VM_OPTIONS -m com.vistamaresoft.treni/com.vistamaresoft.treni.Main "$@"

The generated image DOES contain the native libraries: I tried generating an image WITHOUT the native dependencies and the result is shorter roughly of the same size of those JAR's. The difference, as expected, is in the resulting lib/modules file. So, this file presumably does contain the needed library/ies, but the executable(s) are not able to retrieve them?

I have Googled and 'stackoverflow-ed' for a whole day and I found nothing: any suggestion, help, hint about what it is happening is welcome.

This is the (slightly shortened for brevity) pom.xml;

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

<!-- Several descriptive tags removed for brevity -->
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <target.name>treni</target.name>
        <!-- Versions -->
        <project.version>0.0.1</project.version>
        <java.version>17</java.version>
        <joml.version>1.10.5</joml.version>
        <joml-primitives.version>1.10.4</joml-primitives.version>
        <lwjgl.version>3.3.2</lwjgl.version>
        <imgui-java.version>1.86.10</imgui-java.version>
        <exec-maven-plugin.version>3.0.0</exec-maven-plugin.version>
        <maven-compiler-plugin.version>3.11.0</maven-compiler-plugin.version>
        <maven-dependency-plugin.version>3.6.0</maven-dependency-plugin.version>
        <maven-jar-plugin.version>3.3.0</maven-jar-plugin.version>
        <maven-jlink-plugin.version>3.1.0</maven-jlink-plugin.version>
        <maven-resources-plugin.version>3.3.1</maven-resources-plugin.version>
    </properties>

    <profiles>
        <profile>
            <id>lwjgl-natives-linux-amd64</id>
            <activation>
                <os>
                    <family>unix</family>
                    <arch>amd64</arch>
                </os>
            </activation>
            <properties>
                <lwjgl.natives>natives-linux</lwjgl.natives>
                <imgui.native>natives-linux</imgui.native>
                <imgui.native.module>linux</imgui.native.module>
            </properties>
        </profile>
        <!-- Windows and MacOs profiles removed for brevity -->
    </profiles>

    <dependencies>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl</artifactId>
            <version>${lwjgl.version}</version>
        </dependency>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl-glfw</artifactId>
            <version>${lwjgl.version}</version>
        </dependency>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl-opengl</artifactId>
            <version>${lwjgl.version}</version>
        </dependency>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl-stb</artifactId>
            <version>${lwjgl.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>2.0.7</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-jdk14</artifactId>
            <version>2.0.7</version>
        </dependency>
        <dependency>
            <groupId>org.joml</groupId>
            <artifactId>joml</artifactId>
            <version>${joml.version}</version>
        </dependency>
        <dependency>
            <groupId>io.github.spair</groupId>
            <artifactId>imgui-java-binding</artifactId>
            <version>${imgui-java.version}</version>
        </dependency>

<!-- Natives -->

        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl</artifactId>
            <version>${lwjgl.version}</version>
            <classifier>${lwjgl.natives}</classifier>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl-glfw</artifactId>
            <version>${lwjgl.version}</version>
            <classifier>${lwjgl.natives}</classifier>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl-opengl</artifactId>
            <version>${lwjgl.version}</version>
            <classifier>${lwjgl.natives}</classifier>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl-stb</artifactId>
            <version>${lwjgl.version}</version>
            <classifier>${lwjgl.natives}</classifier>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.github.spair</groupId>
            <artifactId>imgui-java-${imgui.native}</artifactId>
            <version>${imgui-java.version}</version>
            <scope>runtime</scope>
        </dependency>

    </dependencies>

    <build>
        <pluginManagement>
            <plugins>

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>${maven-compiler-plugin.version}</version>
                    <configuration> <!--
                        <release>S${java.version}</release> -->
                        <source>${java.version}</source>
                        <target>${java.version}</target>
                        <showDeprecation>true</showDeprecation>
                    </configuration>
                </plugin>

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-jar-plugin</artifactId>
                    <configuration>
                        <archive>
                            <manifest>
                                <addClasspath>true</addClasspath>
                                <mainClass>com.vistamaresoft.treni.Main</mainClass>
                            </manifest>
                        </archive>
                        <outputDirectory>${project.build.directory}/${target.name}</outputDirectory>
                    </configuration>
                </plugin>

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-javadoc-plugin</artifactId>
                    <version>3.5.0</version>
                    <!-- run with `mvn javadoc:javadoc` -->
                    <configuration>
                        <reportOutputDirectory>${project.basedir}/doc</reportOutputDirectory>
                        <show>public</show>
                    </configuration>
                </plugin>

                <plugin>
                    <groupId>org.moditect</groupId>
                    <artifactId>moditect-maven-plugin</artifactId>
                    <version>1.0.0.Final</version>
                        <configuration>
                            <outputDirectory>${project.build.directory}/modules</outputDirectory>
                            <overwriteExistingFiles>true</overwriteExistingFiles>
                            <modules>
                                <module>
                                    <artifact>
                                        <groupId>io.github.spair</groupId>
                                        <artifactId>imgui-java-binding</artifactId>
                                        <version>${imgui-java.version}</version>
                                    </artifact>
                                    <moduleInfoFile>${project.basedir}/src/main/moduleInfos/imgui.binding.module-info.java</moduleInfoFile>
                                </module>

                                <module>
                                    <artifact>
                                        <groupId>io.github.spair</groupId>
                                        <artifactId>imgui-java-${imgui.native}</artifactId>
                                        <version>${imgui-java.version}</version>
                                    </artifact>
                                    <moduleInfoFile>${project.basedir}/src/main/moduleInfos/imgui.natives.${imgui.native.module}.module-info.java</moduleInfoFile>
                                </module>
                            </modules>
                        </configuration>
                </plugin>

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-jlink-plugin</artifactId>
                    <version>${maven-jlink-plugin.version}</version>
                    <!-- run with `mvn jlink:jlink` separately from `mvn package` as combining both into `mvn package jlink:jlink` raises an error -->
                    <extensions>true</extensions>
                    <configuration>
                        <!-- Module paths overriding the Mavem local repo path for non-modular JAR, modularised with moditect-maven-plugin -->
                        <modulePaths>
                            <modulePath>${project.build.directory}/modules/imgui-java-binding-${imgui-java.version}.jar</modulePath>
                            <modulePath>${project.build.directory}/modules/imgui-java-natives-${imgui.native.module}-${imgui-java.version}.jar</modulePath>
                        </modulePaths>
                        <compress>2</compress>
                        <noHeaderFiles>true</noHeaderFiles>
                        <noManPages>true</noManPages>
                        <stripDebug>true</stripDebug>
                        <launcher>treni=com.vistamaresoft.treni/com.vistamaresoft.treni.Main</launcher>
                    </configuration>
                </plugin>

            </plugins>
        </pluginManagement>

        <!-- PLUG-IN EXECUTIONS -->

        <plugins>
            <plugin>
                <groupId>org.moditect</groupId>
                <artifactId>moditect-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-module-infos</id>
                        <phase>generate-resources</phase>
                        <goals>
                        <goal>add-module-info</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

And this is the module-info.java of the app (currently in a single module):

module com.vistamaresoft.treni
{
    exports             com.vistamaresoft.treni;
    exports             com.vistamaresoft.treni.engine;
    exports             com.vistamaresoft.treni.engine.elements;
    exports             com.vistamaresoft.treni.objectviewer;
    exports             com.vistamaresoft.treni.sim;

    requires            imgui.binding;
    requires transitive org.joml;
    requires            org.lwjgl;
    requires            org.lwjgl.glfw;
    requires            org.lwjgl.opengl;
    requires            org.lwjgl.stb;
    requires transitive org.slf4j;
    requires java.prefs;
    requires java.base;
    requires java.logging;
}

Answer 1

I am far from sure this is the best solution, as it is rather cumbersome, but it is the only one I found so far, it works (mostly) and may be useful to someone else too. I want to thank SPASI in the LWJGL forum, who greatly helped me in finding it.

1. Add native libraries modules to module-info.java: this ensures that the libraries will be found within the runnable image generated by the jlink pug-in. In practice, add to module-info.java one line like

requires transitive org.lwjgl.natives;

for each native library.

2. Unfortunately, both Eclipse and the Maven compiler seem unable to locate these modules within the local Maven repo .m2. The solution is three-fold:

2a) Copy the dependencies from the repo to a local folder, for instance target/modules using the copy-dependencies goal of the Maven dependency plug-in; this must be done before the Maven compile phase (I use the generate-resources phase), for instance:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>3.6.0</version>
    <configuration>
        <outputDirectory>${project.build.directory}/modules</outputDirectory>
        <overWriteReleases>false</overWriteReleases>
        <overWriteSnapshots>false</overWriteSnapshots>
        <overWriteIfNewer>true</overWriteIfNewer>
    </configuration>
    <executions>
        <execution>
            <id>copy-dependencies</id>
            <phase>generate-resources</phase>
            <goals>
                <goal>copy-dependencies</goal>
            </goals>
        </execution>
    </executions>
</plugin>

2b) Maven compiler plug-in: point this plug-in to the folder where the dependencies have been copied, adding this directive to the plug-in configuration:

<compilerArgs>
    <arg>--module-path</arg>
    <arg>${project.build.directory}/modules</arg>
</compilerArgs>

2c) Maven jlink plug-in: point this plug-in also to the same folder, adding the following directive to the plug-in configuration:

<modulePaths>
    <modulePath>${project.build.directory}/modules</modulePath>
</modulePaths>

Now, Maven should correctly compile the project and jlink a runnable application.

3. Eclipse (and perhaps other Java IDE's too): this leaves unsolved the issue with Eclipse not resolving the native library modules added to the module-info.java in step 1.

This precludes building the project from within Eclipse, which I find very useful at least for debugging. Maybe this can be solved for the libraries whose JAR's are already modular (but I do not know how), but surely it cannot be solved for the JAR's which are NOT modular and have to be modularised (for instance with the org.moditect:moditect-maven-plugin): their modules simply do not exist in any well-known location and are generated out-of-source (this happen to me with the io.github.spair:imgui-java-natives-<platform> which is not modular and is often used with openGL / Vulkan projects).

As I use Maven only via command line, I work-around this preparing (our of the Java source and parallel to the main project pom.xml) 2 versions of the module-info.java, one without and one with the added modules.

Normally the project module-info.java is kept equal to the first, shorter, version; when I have to launch a Maven build, a small shell script overwrites module-info with the extended version, run the required mvn command(s) and finally overwrites module-info.java back with the 'normal' version. Not really fool-proof, but usable.