Symbol not found but was included during linking

Question

I've just built magic VLSI software on MacOs Sierra 10.12.2. This includes building tclmagic.dylib library. Now when testing Magic Exec I get:

magic
dyld: lazy symbol binding failed: Symbol not found: _HashInit
  Referenced from: /usr/local/lib/magic/tcl/tclmagic.dylib
  Expected in: flat namespace`

I searched and found the HashInit function inside an utility library which is included during linking:

gcc -g  -I/usr/X11/include  -I/Library/Frameworks/Tk.framework/Versions/8.6/Headers -I/Library/Frameworks/Tcl.framework/Versions/8.6/Headers -fno-common -Wimplicit-int -fPIC  -I/Library/Frameworks/Tk.framework/Versions/8.6/Headers -I/Library/Frameworks/Tcl.framework/Versions/8.6/Headers -I. -I..  -o tclmagic.dylib  -dynamiclib -flat_namespace -undefined suppress -noprebind  \
    ../cmwind/libcmwind.o ../commands/libcommands.o ../database/libdatabase.o ../dbwind/libdbwind.o ../drc/libdrc.o ../debug/libdebug.o ../extract/libextract.o ../graphics/libgraphics.o ../select/libselect.o ../textio/libtextio.o ../tiles/libtiles.o ../windows/libwindows.o ../wiring/libwiring.o ../resis/libresis.o ../sim/libsim.o ../netmenu/libnetmenu.o ../plow/libplow.o ../utils/libutils.o ../ext2sim/libext2sim.o ../ext2spice/libext2spice.o ../calma/libcalma.o ../cif/libcif.o ../plot/libplot.o ../lef/liblef.o ../extflat/libextflat.o ../garouter/libgarouter.o   ../mzrouter/libmzrouter.o ../router/librouter.o     ../irouter/libirouter.o ../grouter/libgrouter.o     ../gcr/libgcr.o ../tcltk/libtcltk.o -lc -lX11 -lGL -lGLU    -lm   -L/usr/X11/lib  -lm

The HashInit function resides inside ../utils/libutils.o. I've also used nm -gU on the two binaries and the symbol is found:

nm -gU utils/libutils.o | grep HashInit
0000000000002880 T _HashInit
00000000000028c0 T _HashInitClient

nm -gU magic/tclmagic.dylib| grep HashInit
000000000011ec70 T _HashInit
000000000011ecb0 T _HashInitClient

I've now tried a separate linking command and dump out a symbol mapping file:

ld  -o tclmagic.dylib -dylib -flat_namespace -undefined suppress -noprebind          ../cmwind/libcmwind.o ../commands/libcommands.o ../database/libdatabase.o ../dbwind/libdbwind.o ../drc/libdrc.o ../debug/libdebug.o ../extract/libextract.o ../graphics/libgraphics.o ../select/libselect.o ../textio/libtextio.o ../tiles/libtiles.o ../windows/libwindows.o ../wiring/libwiring.o ../resis/libresis.o ../sim/libsim.o ../netmenu/libnetmenu.o ../plow/libplow.o ../utils/libutils.o ../ext2sim/libext2sim.o ../ext2spice/libext2spice.o ../calma/libcalma.o ../cif/libcif.o ../plot/libplot.o ../lef/liblef.o ../extflat/libextflat.o ../garouter/libgarouter.o     ../mzrouter/libmzrouter.o ../router/librouter.o     ../irouter/libirouter.o ../grouter/libgrouter.o     ../gcr/libgcr.o ../tcltk/libtcltk.o -lc -lX11 -lGL -lGLU    -lm   -L/usr/X11/lib  -lm  -macosx_version_min 10.12  -all_load -why_load -map debug_map
ld: warning: option -noprebind is obsolete and being ignored

Again, I see the symbol is available:

cat debug_map | grep HashInit
0x0011F390  0x00000040  [ 18] _HashInit
0x0011F3D0  0x00000150  [ 18] _HashInitClient
0x001DADFE  0x00000006  [ 18] _HashInit
0x001DAE04  0x00000006  [ 18] _HashInitClient
0x001DF2B8  0x0000000A  [ 18] _HashInit
0x001DF2C2  0x0000000A  [ 18] _HashInitClient
0x0020C1D8  0x00000008  [ 18] _HashInit
0x0020C1E0  0x00000008  [ 18] _HashInitClient`

Btw, I've seen similar questions here and here, but I believe their scenario may be slightly different as one was cause by cmake mistake and the other was worked around by using DYLD_INSERT_LIBRARIES, which doesn't have an impact on my case.

Thanks in advance, Ronald

score 0 · Answer 1 · answered Jan 25 '17 at 12:16

It seems that adding -exported_symbol_list symbol.list to the linking command will get me pass the issue (symbol.list contains 1 exported symbol each line). However, I'm still getting the following (below command runs in tclsh):

(magic) 1 % load -lazy ./tclmagic.dylib
cannot find symbol "Tclmagic_Init": dlsym(0x10061e9f0, Tclmagic_Init): symbol not found

score 0 · Answer 2 · answered Jan 30 '17 at 16:26

I believe that the answer is that the magic Makefile is not using the Tcl "stubs" libraries. I successfully converted netgen to use stubs, just need to do the same for magic. Using "-lazy" was, indeed, the lazy way to get around the issue, but using the stubs libraries are the correct way.

As a general procedure, file defs.mak needs to define LIB_SPECS with "-ltkstub8.6" and "-ltclstub8.6" instead of "-ltk8.6" and "-ltcl8.6". The fix is more complicated for use with "configure" since defs.mak is derived from defs.mak.in. But I think that a quick change to LIB_SPECS in defs.mak and a rebuild (without running configure) will work (probably have to do "make clean" first).

---Tim

Hi Tim! Thanks a lot for replying... (and for maintaining Magic & qflow btw!) ...At some point I figured out that the stub libraries should be added and that I should also add the -DUSE_TCL_STUBS and -DUSE_TK_STUBS defines. However I'm still getting the dlsym message complaining about Tclmagic_Init... Now I'm running out of Ideas... Regards, Ronald — 20Mhz, Jan 31 '17 at 01:21

Symbol not found but was included during linking

2 Answers2