Hiding symbols in a dynamic library without modifying source code

Question

I have a closed-source 3rd-party shared library I need to link against. Unfortunately, the creators of the 3rd-party library didn't bother to limit which symbols are exported and exported all symbols. The 3rd-party library internally uses an incompatible version of a popular library that I am using in my code but exports conflicting symbols (google's protobuf library). This leads to a runtime error when the protobuffer library version check finds the compile time and run time versions of the library are not compatible. I can solve the problem by reverting to an older version of protobufs 2.3 that matches the version used in the 3rd party library. However, protbuf 2.3 has performance issues that make it unusable slow for my application. I need a way to use protobuf 2.4 in my code and let the 3rd-party library use it's own internal v 2.3.

Is there a way to generate a new version of the 3rd party library that doesn't export the symbols from the protobuf v 2.3 library used internally given only the so file? If I had the source, it would be an easier problem. It seems that tools like objcopy and strip can't actually modify the dynamic symbol table. The only idea I have so far is to create my own shim library that exports only the symbols I need by redirecting calls to the 3rd-party library (opened with dlopen perhaps?).

Is there a better solution?

Linux (fedora, ubuntu)... My preference for affordable scientific / distributed computing... ;-) — heathbar, Apr 18 '12 at 22:05

score 5 · Accepted Answer · answered Apr 19 '12 at 04:42

I found a solution that works... I created a shim library that redirects calls to the third-party library allowing code outside the library to see protbuf v2.4 symbols while code inside the third-party library sees protobuf v2.3 symbols. This workaround was based on the idea posted here: http://www.linuxjournal.com/article/7795

I had to modify the dlopen flags to include RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND. The RTLD_LOCAL flag keeps symbols inside the third-party library from being seen outside the shim library (prevents symbols leaking out). The RTLD_DEEPBIND forces calls from inside the 3rd party library to see only the internal version of the symbols (keeps symbols from leaking in).

To be concrete, here is an example excerpt from my shim library.

#include <stdio.h>
#include <stdint.h>
#include <dlfcn.h>
#include "libhdfs/hdfs.h"

//#define PRINT_DEBUG_STUFF

// Helper function to retrieve a function pointer to a function from libMapRClient
// while isolating the symbols used internally from those already linked externaly
// to workaround symbol collision problem with the current version of libMapRClient.
void* GetFunc(const char* name){
  #ifdef PRINT_DEBUG_STUFF
    printf("redirecting %s\n", name);
  #endif
  void *handle;
  char *error;

  handle = dlopen("/opt/mapr/lib/libMapRClient.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

  if (!handle) {
    fputs(dlerror(), stderr);
    exit(1);
  }
  void* fp = dlsym(handle, name);
  if ((error = dlerror()) != 0) {
    fprintf(stderr, "%s\n", error);
    exit(1);
  }
  return fp;
}


hdfsFS hdfsConnect(const char* host, tPort port) {
  typedef hdfsFS (*FP) (const char* host, tPort port);
  static FP ext = 0;
  if (!ext) {
    ext = (FP)GetFunc("hdfsConnect");
  }
  return ext(host, port);
}


int hdfsCloseFile(hdfsFS fs, hdfsFile file) {
  typedef int (*FP) (hdfsFS fs, hdfsFile file);
  static FP ext = 0;
  if (!ext) {
    ext = (FP)GetFunc("hdfsCloseFile");
  }
  return ext(fs, file);
}

... and so on for the other public API functions

Isn't the shim basically just hiding the dlopen / dlsym nastiness from the main application? It doesn't seem necessary. — Trebor Rude, Apr 20 '16 at 19:37

score 0 · Answer 2 · answered Apr 18 '12 at 18:37

0

It might help to change the order of libraries you pass to the linker. If several libraries export the same symbol the linker should use the one from the first library. But, not all linkers are according to that, so have a look into its documentation.

answered Apr 18 '12 at 18:37

bjhend

1,538
11
25

Thanks for this idea... I tried this, it doesn't quite work because it forces all call sites to use the same version of the symbol (either from version 2.3 or 2.4). However, I need calls within the 3rd party library to call v2.3 of the symbol and calls within my code to call v2.4 of the symbol. Otherwise the library detects mismatched versions and aborts. Switching the library link order only changes if the abort happens when my code loads the library or the 3rd party code loads the library. – heathbar Apr 18 '12 at 19:28
So you need a kind of partial linking. In the first stage link your code to v2.4 and independently 3rd party lib's code to v2.3 and in the second stage link the results together. Maybe there are linker options which allow such incomplete linkings of the first stage? – bjhend Apr 18 '12 at 20:58

Hiding symbols in a dynamic library without modifying source code

2 Answers2