0

I have some misunderstanding about esp pointer.

below are the code which shown in one of previous exams. The returned value is 1.

func:   xor eax,eax
        call L3
        L1: call dword[esp]
        inc eax
        L2: ret
        L3: call dword[esp]
        L4: ret

Now, I will explain how I think and hope someone will correct me or approve. This is how I think when I know what is the answer so I am not sure I`m thinking correctly at all.

  1. eax = 0
  2. We push to stack return address which is the next line, i.e label L1.
  3. We jump to L3.
  4. We push to stack return address which is the next line, i.e label L4.
  5. We jump to L1.
  6. We push to stack return address which is the next line, i.e inc eax.
  7. We jump to L4.
  8. We jump to line where is inc eax and stack is now empty.
  9. eax = 1.
  10. we end here(at label L2) and return 1.
Rozen Vector
  • 11
  • 2
  • why question is down-voted? please let me know, I will improve it. – Rozen Vector Aug 14 '20 at 15:01
  • I think `eax` = 2 and the caller of func is called once from the instruction at L1. – ecm Aug 14 '20 at 15:21
  • 1
    You need to keep going after step 10. Where does 10. return to? – Erik Eidt Aug 14 '20 at 15:26
  • I think I miss something, step 10 is the end. It means - return via eax register to function who call this func. – Rozen Vector Aug 14 '20 at 17:14
  • 1
    `retn` means "pop the dword off the stack's top into `eip`". It does not unconditionally "return to function which called this function", it returns to whatever address is on top of the stack. Also, your step 8 "stack is now empty" comment is wrong. – ecm Aug 14 '20 at 17:23
  • 1
    This isn't a valid function; it can't be said to "return" because it eventually *calls* its return address, jumping there with 2 extra dwords on the stack (including the original return address) vs. if it had returned. – Peter Cordes Aug 14 '20 at 18:40
  • The key thing you’re missing is that `call [esp]` doesn’t remove the address from the stack. – prl Aug 14 '20 at 19:50

1 Answers1

1

I think eax = 2 and the caller of func is called once from the instruction at L1. In the following I will trace execution to show you what I mean.

I re-arranged your example a bit to make it more readable. This is NASM source. I think this should be equivalent to the original (assuming the D bit and B bit are set, i.e. you're running in normal 32-bit mode).

        bits 32
func:
        xor eax, eax
        call L3
.returned:
L1:
        call near [esp]
.returned:
        inc eax
L2:
        retn
L3:
        call near [esp]
.returned:
L4:
        retn

Now assume we start with some function that does this:

foo:
        call func
.returned:
        X
        retn

This is what happens:

  1. At foo we call func. The address of foo.returned is put on the stack (say, stack slot -1).

  2. At func we set eax to zero.

  3. Next we call L3. The address of func.returned = L1 is put on the stack (slot -2).

  4. At L3 we call the dword on top of the stack. Here this is func.returned. The address of L3.returned = L4 is put on the stack (slot -3).

  5. At L1 we call the dword on top of the stack. Here this is L3.returned. The address of L1.returned is put on the stack (slot -4).

  6. At L4 we return. This pops L1.returned (from slot -4) into eip.

  7. At L1.returned we do inc eax, setting eax to 1.

  8. Then at L2 we return. This pops L3.returned (from slot -3) into eip.

  9. At L4 we return. This pops func.returned (from slot -2) into eip.

  10. At L1 we call the dword on top of the stack. Here this is foo.returned. The address of L1.returned is put on the stack (slot -2).

  11. At foo.returned we execute whatever is there which I marked X. Assuming that the function returns using a retn eventually then...

  12. ... we return. This pops L1.returned (from slot -2) into eip.

  13. At L1.returned we do inc eax. Assuming that X did not alter eax then we now have eax = 2.

  14. Then at L2 we return. This pops foo.returned (from slot -1) into eip.

If my assumptions are correct then eax is 2 in the end.

Note that it is really strange to call the return address on top of the stack. I cannot imagine a practical use for this.

Also note that if, in a debugger, you proceed-step the call to func in foo then at step 11 the debugger might return control to the user, with eax equal to 1. However, the stack is not balanced at this point.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
ecm
  • 2,583
  • 4
  • 21
  • 29
  • 1
    You're right, if you throw this into `_start` as a static executable to single-step it, at step `10` the `call [esp]` uses the incoming top-of-stack as a call target. In this case it's `argc` (1), so it faults there, with EAX=1. So this code isn't a ABI-compliant function, and there's no real way to say what it does! If called from a caller that does `leave` or similar after the call, the extra return address will be discarded, otherwise it breaks the stack (e.g. `pop ebx`). It looked interesting, but turns out it's a pretty poor exam question. – Peter Cordes Aug 14 '20 at 18:36