3

I have an assignment that asks me to find the return value of main though register inspection (we're learning gdb), how would I go about doing that?

Doug Smith
  • 29,668
  • 57
  • 204
  • 388
  • 1
    I am not saying duplicate because you ask for register inspection specifically, but this is almost the same (and one answer actually explains the register inspection method): http://stackoverflow.com/questions/267674/inspect-the-return-value-of-a-function-in-gdb – jogojapan Apr 01 '13 at 02:08

1 Answers1

6

In general, look at the calling convention.

In x86, all calling conventions return small integer results on EAX, and large (64-bit) results on EDX:EAX (EDX holding the higher bits), and floating point results in FP0.

In x64, small integer results are returned on RAX, and floating point results in FP0.

In ARM (including thumb-mode), integer results are returned in R0.

If you're trying to work out where to put your breakpoint, my suggestion is to put a breakpoint at the start of the main function. If you do that, the return address (i.e. where main will return to once it is finished executing) will be the value on the top of the stack. If you put a breakpoint there, you'll break just after the main function has finished executing.

Since main has the return type of int, you can look at EAX (or RAX or R0) to see what value main returned.

SecurityMatt
  • 6,593
  • 1
  • 22
  • 28
  • Nice explanation. However, I'm inclined to simply try and guide people in the right direction when they specifically ask questions from assignments rather than solve the problem for them. – dcow Apr 01 '13 at 02:02
  • If I set a breakpoint at the line `int main() {` it just breaks at the next line where code is actually being executed. And when you say where it will return to, does that mean where the value the function returns will return to? And I get this when I run `info registers`: `rax 0x100000ec0 4294971072` what exactly does that mean? – Doug Smith Apr 01 '13 at 02:43
  • @DougSmith: If you have symbols and source code, put a breakpoint on the return keyword at the end of the main function and step through until you see the "ret". Look at RAX at *that* point. – SecurityMatt Apr 01 '13 at 02:59
  • As in the line return is found on (say `b 18`)? – Doug Smith Apr 01 '13 at 03:08