1

I came across below code for walking backtrace

struct stack_frame {
  struct stack_frame *prev;
    void *return_addr;
} __attribute__((packed));
typedef struct stack_frame stack_frame;

__attribute__((noinline, noclone))
void backtrace_from_fp(void **buf, int size)
{
    int i;
    stack_frame *fp;

    __asm__("movl %%ebp, %[fp]" :  /* output */ [fp] "=r" (fp));

    for(i = 0; i < size && fp != NULL; fp = fp->prev, i++)
        buf[i] = fp->return_addr;
}

the reason behind looking for this code is we are using a 3rd party malloc hook hence don't want to use backtrace which again allocates memory. Above doesn't work for x86_64 and I modified asm statement to 

    __asm__("movl %%rbp, %[fp]" :  /* output */ [fp] "=r" (fp));

I get crash

(gdb) bt
#0  backtrace_from_fp (size=10, buf=<optimized out>) at src/tcmalloc.cc:1910
#1  tc_malloc (size=<optimized out>) at src/tcmalloc.cc:1920
#2  0x00007f5023ade58d in __fopen_internal () from /lib64/libc.so.6
#3  0x00007f501e687956 in selinuxfs_exists () from /lib64/libselinux.so.1
#4  0x00007f501e67fc28 in init_lib () from /lib64/libselinux.so.1
#5  0x00007f5029a32503 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#6  0x00007f5029a241aa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#7  0x0000000000000001 in ?? ()
#8  0x00007fff22cb8e24 in ?? ()
#9  0x0000000000000000 in ?? ()
(gdb)
(gdb) p $rbp
$2 = (void *) 0x7f501e695f37
(gdb) p (stack_frame *)$rbp
$3 = (stack_frame *) 0x7f501e695f37
(gdb) p *$3
$4 = {prev = 0x69662f636f72702f, return_addr = 0x6d6574737973656c}
(gdb) x /1xw 0x69662f636f72702f
0x69662f636f72702f:     Cannot access memory at address 0x69662f636f72702f
(gdb) fr
#0  backtrace_from_fp (size=10, buf=<optimized out>) at src/tcmalloc.cc:1910
1910    in src/tcmalloc.cc
(gdb)

Am I missing something ?. Any help on how can I reconstruct the same via code ?.

BalajiKK
  • 11
  • 1
  • 1
  • movq is needed in 64-bit mode instead of movl: `__asm__("movq %%rbp, %[fp]" : /* output */ [fp] "=r" (fp));` – Renat Mar 12 '20 at 10:49
  • 1
    If you are using gcc, you can use the various [return address builtins](https://gcc.gnu.org/onlinedocs/gcc/Return-Address.html). – Jester Mar 12 '20 at 11:21
  • 3
    You cannot rely on `rbp` being the frame pointer, because sane software doesn't *use* frame pointers any more. Frame pointers are a relic of (bad) assembly programing and bad compilers, with bad debug info. There is no *need* for frame pointers when you have reasonably good tooling, so modern toolchains instead use the `rbp` register as a general-purpose register, which speeds up the code without any relevant downside. – EOF Mar 12 '20 at 17:36
  • @Jester Even I want to do the same, but the problem is I am not sure how many frames are there and hence calling this builtin with an index not present results in crashing. – BalajiKK Mar 13 '20 at 02:56
  • It would normally return 0 before crashing though, just as your very own `fp != NULL` condition. – Jester Mar 13 '20 at 03:01
  • @Renat I changed the same but still not able to reconstruct the backtrace, I see the same working with my sample program, but when I build this with my product I hit this crash. – BalajiKK Mar 13 '20 at 04:24

1 Answers1

3

Am I missing something ?

The code you referenced assumes the compiled code is using frame pointer register chain.

This was the default on (32-bit) i*86 up until about 5-7 years ago, and has not been the default on x86_64 since ~forever.

The code will most likely work fine in non-optimized builds, but will fail miserably with optimization on both 32-bit and 64-bit x86 platforms using non-ancient versions of the compiler.

If you can rebuild all code (including libc) with -fno-omit-frame-pointer, then this code will work most of the time (but not all the time, because libc may have hand-coded assembly, and that assembly will not have frame pointer chain).

One solution is to use libunwind. Unfortunately, using it from inside malloc can still run into a problem, if you (or any libraries you use) also use dlopen.

Employed Russian
  • 199,314
  • 34
  • 295
  • 362
  • I would guess that all of glibc's hand-written asm functions are leaf functions; they aren't part of anything else's call chain via callbacks, and they don't call `malloc`. examples include `memcpy`, `strchr`, and a few FP math functions for some ISAs. – Peter Cordes Mar 13 '20 at 03:38
  • re: avoiding malloc: perhaps you could compile `libunwind` to use a custom allocator that uses raw `mmap` directly, if it doesn't have to be fast. Or a separate simple malloc/free. As long as its allocations don't need to be compatible with the real `free`. – Peter Cordes Mar 13 '20 at 03:40
  • 1
    @PeterCordes I vaguely remember hand-written assembly functions calling initializers (which can then call anything). And `libunwind` already doesn't call `malloc`. But it has to call `dl_iterate_phdr`, which requires the loader lock, when may be acquired before malloc lock. Gory details: https://lists.nongnu.org/archive/html/libunwind-devel/2010-05/msg00006.html – Employed Russian Mar 13 '20 at 04:06
  • Maybe just selection bias on my part from having only looked at performance-relevant but simple functions. I can't think of any that would need to init a lookup table but are written in asm; e.g. the char classification functions like `isalnum` are written in C. But sure, could be something. – Peter Cordes Mar 13 '20 at 04:21