dlsym ends in infinite loop

Question

I am trying to build a Linux library(*.so) to use it in a Java application. This library itself loads an dll-file with native functions.

This is my C++ code:

__delspec(dllexport) void __cdecl GetDllVersion(void){
    typedef int(*GetDllVersion)(int*,int*,int*,int*);

    void* lib = dlopen("~/lib.dll",RTLD_NOW);
    cout << "Loading Symbol..." << endl;
    GetDllVersion getVer=  (GetDllVersion) dlsym(lib,"GetDllVersion");

    dlclose(lib);

}

The code was compiled with wineg++ -shared lib.cpp -o libexports.so -Wl,--no-as-needed -ldl without errors.

The Java application prints out "Loading Symbol..." in a loop and then terinates without any message. I could determine that this has to do something with dlsym().

With nm -D lib.so I could look inside the lib.so. The function GetDllVersion() is indeed a symbol inside this library.

Can somebody tell me why there is a infinite loop and wyh the Java VM is terminating?

Regards Wurmi

score 0 · Answer 1 · answered Oct 26 '15 at 03:30

This line:

void* lib = dlopen("~/lib.dll",RTLD_NOW);

will always fail, because dlopen does not do tilde-expansion (in general, only shell does). You really should check dlopen return value.

This line:

GetDllVersion getVer=  (GetDllVersion) dlsym(lib,"GetDllVersion");

is equivalent to dlsym(RTLD_DEFAULT, ...) (because RTLD_DEFAULT == 0 and lib == NULL) and as such returns you a pointer to the function you are already in, resulting in infinite recursion, and eventual crash due to stack exhaustion.

dlsym ends in infinite loop

1 Answers1