Java: load shared libraries with dependencies

Question

I am wrapping a shared library (written in C) with Java using JNA. The shared library is written internally, but that library uses functions from another external library, which again depends another external library. So the situation is something like this:

ext1 <- ext2 <- internal

I.e. the internal uses external library ext2 which again uses external library ext1. What I have tried is:

System.loadLibrary("ext1");
System.loadLibrary("ext2");
NativeLIbrary.loadLibrary("internal",xxx.class);  

This approach fails with "UnresolvedException" when loading the library "ext2"; the linker complains about symbols which are indeed present in the library "ext1". So it semmes that the System.loadLibrary() function does not make the symbols from "ext1" globally available? When using the stdlib function dlopen() as:

handle = dlopen( lib_name , RTLD_GLOBAL );

All the symbols found in @lib_name will be available for symbol resolution in subsequent loads; I guess what I would like was something similar for the java variety System.loadLibrary()?

Regards - Joakim Hove

Answer 1

It's an old question, but I've found an acceptable solution, which should also be portable, and I thought I should post an answer. The solution is to use JNA's NativeLibrary#getInstance(), because on Linux this will pass RTLD_GLOBAL to dlopen() (and on Windows this is not needed).

Now, if you are using this library to implement a Java native method, you will also need to call System.load() (or Sysem.loadLibrary()) on the same library, after calling NativeLibrary#getInstance().

First, a link to a JNA bug: JNA-61

A comment in there says that basically one should load dependencies before the actual library to use from within JNA, not the standard Java way. I'll just copy-paste my code, it's a typical scenario:

String libPath =
        "/path/to/my/lib:" + // My library file
        "/usr/local/lib:" +  // Libraries lept and tesseract
        System.getProperty("java.library.path");

System.setProperty("jna.library.path", libPath);

NativeLibrary.getInstance("lept");
NativeLibrary.getInstance("tesseract");
OcrTesseractInterf ocrInstance = (OcrTesseractInterf)
        Native.loadLibrary(OcrTesseractInterf.JNA_LIBRARY_NAME, OcrTesseractInterf.class);

I've written a small library to provide OCR capability to my Java app using Tesseract. Tesseract dependes on Leptonica, so to use my library, I need to load libraries lept and tesseract first. Loading the libraries with the standard means (System.load() and System.loadLibrary()) doesn't do the trick, neither does setting properties jna.library.path or java.library.path. Obviously, JNA likes to load libraries its own way.

This works for me in Linux, I guess if one sets the proper library path, this should work in other OSs as well.

Answer 2

There is yet another solution for that. You can dlopen directly inside JNI code, like this:

void loadLibrary() {
  if(handle == NULL) {
    handle = dlopen("libname.so", RTLD_LAZY | RTLD_GLOBAL);
    if (!handle) {
      fprintf(stderr, "%s\n", dlerror());
      exit(EXIT_FAILURE);
    }
  }
}

...
...

loadLibrary();

This way, you will open library with RTLD_GLOBAL.

You can find detailed description here: http://www.owsiak.org/?p=3640

Answer 3

OK;

I have found an acceptable solution in the end, but not without significant amount of hoops. What I do is

1. Use the normal JNA mechanism to map the dlopen() function from the dynamic linking library (libdl.so).
2. Use the dlopen() function mapped in with JNA to load external libraries "ext1" and "ext2" with the option RTLD_GLOBAL set.

It actually seems to work :-)

Answer 4

As described at http://www.owsiak.org/?p=3640, an easy but crude solution on Linux is to use LD_PRELOAD.

If that's not acceptable, then I'd recommend the answer by Oo.oO: dlopen the library with RTLD_GLOBAL within JNI code.

Answer 5

Try this, add this function to your code. Call it before you load your dlls. For the parameter, use the location of your dlls.

    public boolean addDllLocationToPath(String dllLocation)
    {
        try
        {
            System.setProperty("java.library.path", System.getProperty("java.library.path") + ";" + dllLocation);
            Field fieldSysPath = ClassLoader.class.getDeclaredField("sys_paths");
            fieldSysPath.setAccessible(true);
            fieldSysPath.set(null, null);
        }
        catch (Exception e)
        {
            System.err.println("Could not modify path");
            return false;
        }
        return true;
    }
}

Answer 6

In order to fix your issue you can use this package: https://github.com/victor-paltz/global-load-library. It loads the libraries directly with the RTLD_GLOBAL flag.

Here is an example:

import com.globalload.LibraryLoaderJNI;

public class HelloWorldJNI {
 
    static {
        // Loaded with RTLD_GLOBAL flag
        try {
            LibraryLoaderJNI.loadLibrary("/path/to/my_native_lib_A");
        } catch (UnsatisfiedLinkError e) {
            System.Println("Couldn't load my_native_lib_A");
            System.Println(e.getMessage());
            e.printStackTrace();
        }

        // Not loaded with RTLD_GLOBAL flag
        try {
            System.load("/path/to/my_native_lib_B");
        } catch (UnsatisfiedLinkError e) {
            System.Println("Couldn't load my_native_lib_B");
            System.Println(e.getMessage());
            e.printStackTrace();
        }
    }
    
    public static void main(String[] args) {
        new HelloWorldJNI().sayHello();
    }
 
    private native void sayHello();
}

It is using the same dlopen() trick as the previous answers, but it is packaged in a standalone code.