2

I have a compiled source code executable which has redefined malloc() with a custom function CustMalloc() using macro substitution.

As seen from the below backtrace, the compiled source code executable is supposed to be dynamically linked with libMRegAccess.so and libusb-1.0.so.0 shared objects. Both these shared objects were independently compiled and they have no means of knowing the above macro substitution during their compilation.

Now, bsd-asprintf.c I presume is a linux source file and the call to asprintf() and vasprintf() are called from the libusb-1.0.so.0 shared object when executing, which in turn tries to call malloc().

I'm unable to understand why the custom function CustMalloc() is getting called instead of the actual malloc().

FYI, the semaphore that is required in this backtrace is not yet created and hence the crash. The expectation is that the CustMalloc() should not be invoked in this code flow as the call is being made from an independently built shared object.

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 1269]
0x0000007fb7f896cc in __new_sem_wait_fast () from /lib//libpthread.so.0
(gdb) bt
#0  0x0000007fb7f896cc in __new_sem_wait_fast () from /lib//libpthread.so.0
#1  0x0000007fb7f898fc in sem_wait@@GLIBC_2.17 () from /lib//libpthread.so.0
#2  0x0000000001b09000 in SemTake (SemId=0x0) at <compiled_source_code.c>
#3  0x0000000000d6cffc in ContextLock () at <compiled_source_code.c>
#4  0x0000000000d993e4 in CustMalloc (size=128) at <compiled_source_code.c>
#5  0x0000000001c88a2c in vasprintf (str=0x7fb5eaf5f8, fmt=0x7fb7e8a640 "usb%s", ap=...) at bsd-asprintf.c:61
#6  0x0000000001c88c50 in asprintf (str=0x7fb5eaf5f8, fmt=0x7fb7e8a640 "usb%s") at bsd-asprintf.c:120
#7  0x0000007fb7e853cc in linux_enumerate_device () from /usr/lib/libusb-1.0.so.0
#8  0x0000007fb7e854c4 in sysfs_scan_device () from /usr/lib/libusb-1.0.so.0
#9  0x0000007fb7e85b80 in op_init () from /usr/lib/libusb-1.0.so.0
#10 0x0000007fb7e7dd1c in libusb_init () from /usr/lib/libusb-1.0.so.0
#11 0x0000007fb7ea65fc in cyusb_open(unsigned short, unsigned short) () from /usr/lib/libMRegAccess.so
#12 0x0000007fb7ea33f4 in InitDefaultUSBConn () from /usr/lib/libMRegAccess.so
#13 0x0000007fb7ea58e0 in openDefaultUSBDriver () from /usr/lib/libMRegAccess.so
#14 0x00000000010ddd94 in InitDrv () at <compiled_source_code.c>
#15 ... at <compiled_source_code.c>
#16 ... at <compiled_source_code.c>
#17 ... at <compiled_source_code.c>
#18 ... at <compiled_source_code.c>
#19 0x0000007fb7f80fd0 in start_thread () from /lib//libpthread.so.0
#20 0x0000007fb7d8cf60 in ?? () from /lib//libc.so.6
Chris Petrus
  • 193
  • 1
  • 3
  • 15

1 Answers1

0

I'm unable to understand why the custom function CustMalloc() is getting called instead of the actual malloc()

This appears to be happening because you compiled and linked bsd-asprintf.c (with your macro redefinition) into your main executable.

You can tell that asprintf and CustMalloc are part of your binary, because their addresses are very different from other library routines (such as linux_enumerate_device or sem_wait).

If you want to know where asprintf is defined (which archive library or object file it comes from), relink your executable with -Wl,-y,asprintf flag, and the linker will tell you.

Employed Russian
  • 199,314
  • 34
  • 295
  • 362