Dynamically loaded library remains loaded despite dlclose

Question

Today I'm looking for some enlightenment of the deep magic inside the dynamic loader. I'm debugging/troubleshooting a plugin system for a C++ application running on Linux. It loads plugins via dlopen (RTLD_NOW | RTLS_LOCAL) and releases them using dlclose. Nothing extraordinary - one would think.

However, I noticed that some plugins remain loaded even after dlclose is successfully called^*. I concluded this after looking at the running process' memory map using pmap. Some libs get immediately removed from process memory and others keep lingering around apparently indefinitely.

Continuing on, the dlopen man page states:

The function dlclose() decrements the reference count on the dynamic library handle handle. If the reference count drops to zero and no other loaded libraries use symbols in it, then the dynamic library is unloaded.

That means the problem boils down to these two possibilities; Either the reference counts aren't zero, or other loaded libs are using symbols from some - but not all - of the plugins.

I'm pretty sure (although not 100%) that the reference counts are zero. The application's plugin manager handles all plugins exactly the same. It also makes sure that a plugin doesn't get loaded multiple times. IMO loading & unloading should therefore behave the same for all plugins.

That leaves the second possibility: other loaded libs are using symbols from the plugins. Another typical case of 'That should never happen'. Although it's certainly possible. We're using gcc and default visibility and as far as I've seen nothing is stripped, so tons of symbols are being exported. Actually that worries me much more as these plugins should be independent.

Here then are my open questions at this point:

Are my conclusions so far correct?
Do you know a way to verifydlopen's reference count?
If there are internal symbols of my plugins (accidentally) used by other libs, is there a way to track down who is using which symbols?

My machine is: Linux 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:44 UTC 2014 i686 i686 i686 GNU/Linux

^* I should mention that all of the loading and unloading happens in the main thread, so there should be no multi threading issue here.

_"or other loaded libs are using symbols from some - but not all - of the plugins."_ I'm pretty sure the OS can't detect this, at least not without scanning the process's entire memory space. — Colonel Thirty Two, Apr 10 '15 at 14:09
Use `ltrace` and/or `strace` to understand at least when are shared objects loaded. — Basile Starynkevitch, Apr 10 '15 at 14:13
@ColonelThirtyTwo I agree, hence my confusion :) Perhaps I'm misunderstanding the second clause of the sentnce from dlopen's man page above — djf, Apr 10 '15 at 14:13
Use also `valgrind` or compile with `gcc -fsanitize=address` ; you heap might be corrupted! — Basile Starynkevitch, Apr 10 '15 at 14:14
@BasileStarynkevitch I hadn't considered heap corruption. Thanks for the suggestions. — djf, Apr 10 '15 at 18:28

score 1 · Answer 1 · answered Apr 10 '15 at 15:03

other loaded libs are using symbols from the plugins

If other libs are not linked to that shared library at link time, referring to symbols of a shared library does not prevent that shared library from being unloaded.

To debug the run-time linker set environment variable LD_DEBUG to all, e.g. LD_DEBUG=all ./my_app. See man ld.so for details.

Dynamically loaded library remains loaded despite dlclose

1 Answers1