conflict in symbols exposed by tcmalloc and glibc

Question

I was recently debugging a crash in a product and identified the cause to be a conflict in the memory allocation symbols exposed by glibc and tcmalloc. I wrote the following sample code for exposing this issue:

#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <assert.h>
#include <stdlib.h>

int main()
{
        struct addrinfo hints = {0}, *res = NULL;
        hints.ai_family = AF_UNSPEC;
        hints.ai_socktype = SOCK_STREAM;
        int rc = getaddrinfo("myserver", NULL, &hints, &res);
        assert(rc == 0);
        return 0;
}

I compiled it using the following command:

g++ temp.cpp -g -lresolv

I executed the program using the following command:

LD_PRELOAD=/path/to/libtcmalloc_minimal.so.4 ./a.out

The program crashes with the following stack:

#0 0x00007ffff6c7c875 in *__GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64

#1 0x00007ffff6c7de51 in *__GI_abort () at abort.c:92

#2 0x00007ffff6cbd8bf in __libc_message (do_abort=2, fmt=0x7ffff6d8c460 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:186

#3 0x00007ffff6cc30c8 in malloc_printerr (action=2, str=0x7ffff6d88fec "free(): invalid pointer", ptr=) at malloc.c:6282

#4 0x00007ffff6cc810c in *__GI___libc_free (mem=) at malloc.c:3733

#5 0x00007ffff6839e89 in _nss_dns_gethostbyname4_r (name=0x400814 "myserver", pat=0x7fffffffdfa8, buffer=0x7fffffffd9b0 "myserver.mydomain.com", buflen=1024, errnop=0x7fffffffdfbc, herrnop=0x7fffffffdf98, ttlp=0x0) at nss_dns/dns-host.c:341

#6 0x00007ffff6d11917 in gaih_inet (name=0x400814 "myserver", service=0x7fffffffdf88, req=0x7fffffffe1d0, pai=0x7fffffffe160, naddrs=0x7fffffffe168) at ../sysdeps/posix/getaddrinfo.c:880

#7 0x00007ffff6d14301 in *__GI_getaddrinfo (name=0x400814 "myserver", service=0x0, hints=0x7fffffffe1d0, pai=0x7fffffffe200) at ../sysdeps/posix/getaddrinfo.c:2452

#8 0x00000000004006f0 in main () at temp.cpp:12

The reason for this is that the free() function called by _nss_dns_gethostbyname4_r() from libnss_dns.so is from libc.so while the corresponding malloc() was called from libresolv.so from libtcmalloc_minimal.so. The addresses of tcmalloc's malloc() and free() functions are getting into the GOT of libresolv.so leading to this crash. The crash goes away if I don't link my program to libresolv.so.

Now for my question. Is there any documentation which explains how to safely use tcmalloc to avoid crashes like this ?

Florian Weimer · Accepted Answer · 2018-06-15T11:56:10.723

3

glibc has some documentation for interposing malloc:

Replacing malloc

Something else must be going here, though. Typical builds of glibc and glibc will get this right (even in fairly old versions of either package).

My best guess is you are using some SUSE glibc variant, which uses RTLD_DEEPBIND for NSS modules. This results in a known issue with malloc interposition. SUSE suggests setting the RTLD_DEEPBIND=0 environment variable as a workaround.

edited Jun 15 '18 at 11:56

answered Jun 15 '18 at 11:49

Florian Weimer

32,022
3
48
92

thanks for the pointer ! I am indeed testing on SuSE (SLES11SP4) and the crash didn't happen after setting the environment variable. I didn't expect glibc to be the culprit here and was expecting tcmalloc to have some documentation for this. If I don't get a pointer to tcmalloc documentation for this issue (as I have asked in my question above), I'll accept this answer. – Rahul Jun 15 '18 at 12:43

conflict in symbols exposed by tcmalloc and glibc

1 Answers1