0

I am trying, as an exercise, to make some sort of custom profiler for binaries in C, using the ptrace api. I assume all binaries to be traced have been statically linked, I have access to tools such as nm(1), objdump and readelf and use a Linux, x86, 32 bit system.

In the current phase I am trying to create a dynamic call tree/graph (relative calls only) of the traced process and include the total number of instructions executed in each function call. In order to do that, I tried to:

  • Retrieve all user defined symbols in the ELF file using nm(1) as well as their addresses
  • Use ptrace to step through the code and identify call and ret restructions
  • After each call, use the rip register to figure out the address of the current instruction and within which function this instruction is; thus deducing the corresponding symbol name.

My question is relative to this last point. I was wondering if there is a way, using the ptrace api, to identify the call instruction as well as the address of the function to which the execution will jump; or even better directly the symbol name which corresponds to this function.

I have tried reading the documentation for ptrace but it is, at least for me, far from clear. Is there a standard approach to what I am trying to do? Is my approach maybe completely wrong?

Desperados
  • 434
  • 5
  • 13
  • GCC can generate a profiling version of a binary, which is probably the closest you can come to a "standard approach". (See https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Instrumentation-Options.html#Instrumentation-Options.) But it works by injecting profiling code into function entry and exit blocks. Another common approach is using a high-speed emulator, as with valgrind; trying to singlestep a program with `ptrace` is likely to be too slow to be useful for any non-trivial application. – rici Mar 23 '21 at 23:58
  • Also clang: https://clang.llvm.org/docs/UsersManual.html#id45 – rici Mar 24 '21 at 00:02
  • @rici what about injecting sigtraps at all call and ret instructions. And counting from the beginning of the functions to the end every time execution reaches such an instruction. That sounds a lot faster that single stepping everything. But the point is ti implement it myself (although the info about gcc is actually really interesting) – Desperados Mar 24 '21 at 02:13
  • 1
    How do you know that a function is entered through CALL or left through RET? Tail-call optimisation means that one or both might be JMP. And there are lots of other optimisations, although I guess you could turn all optimisations off (odd way to profile, though :-) ). I'm not sure what "counting from the beginning" means, since most functions have conditional branches which make it difficult to predict what the function will do. – rici Mar 24 '21 at 03:36
  • 1
    If you don't have assistance from the compiler, just finding CALL and RET instructions will be a bit of a challenge, assuming your target is x86. But there are open-source emulators, such as https://www.qemu.org/ and https://sourceforge.net/projects/bochs/. (Valgrind is also open source but I don't know how easy it would be to get at the VM at its core. That might be your best bet, though.) – rici Mar 24 '21 at 03:38

0 Answers0