Assembly language - Calling convention

Question

Can someone give me some real example to help me understand what is the callee and what is the caller in assembly language? I have been through most of the sources, but still I can't get how to distinguish between the callee and the caller. Thank you in advance. (I am adding this explanation - THANK YOU AGAIN! sorry for the code I am new here and do not know how to insert the code to display properly) I am trying to understand this piece of code - learning the basics of assembly. I am trying to add comment for each command (I am sure someone else like me would need it too:

sub_401040      proc near               
.text:00401040
.text:00401040
.text:00401040 000                 push    ebp    ; Push the content of ebp register onto the stack
.text:00401041 004                 mov     ebp, esp    ;Allocating 8 bytes of storage ; move the esp register content into ebp register
.text:00401043 004                 push    ecx
.text:00401044 008                 mov     eax, [ebp+8]
.text:00401047 008                 push    eax    ; Push eax contents onto the stack
.text:00401048 00C                 call    ds:lstrlenA
.text:0040104E 008                 add     eax, [ebp+0Ch]
.text:00401051 008                 mov     edx, eax
.text:00401053 008                 mov     ecx, 1    ;Set buffer address
.text:00401058 008                 call    sub_401000
.text:0040105D 008                 mov     [ebp-4], eax
.text:00401060 008                 mov     ecx, [ebp-4]
.text:00401063 008                 shl     ecx, 2
.text:00401066 008                 mov     [ebp-4], ecx
.text:00401069 008                 mov     edx, [ebp-4]
.text:0040106C 008                 push    edx
.text:0040106D 00C                 push    offset aResultD ; "Result: %d\n"
.text:00401072 010                 call    ds:printf
.text:00401078 010                 add     esp, 8    ; clean up the stack by adding the size of the argument to esp register
.text:0040107B 008                 mov     eax, 539h
.text:00401080 008                 mov     esp, ebp
.text:00401082 004                 pop     ebp    ;Restore old frame pointer
.text:00401083 000                 retn        ; Return near
.text:00401083     sub_401040      endp

I read that, in order to define the calling convention I need to figure out who is the caller and who is the callee :) I spent so much time to understand the logic: is call ds:lstrlenA means is the callee? and the sub_401040 is the caller? and can we have more than one calling convention in one program? like cdecl along with stdcall? I am not a programmer, and I am not writing a code, I just want to understand how this works to help analysing viruses.

Answer 1

The caller is the one with the relevant call (or rcall/blx/jalr/etc. function calling instruction), and the callee is the function that is being called.

func:
    do_stuff

func2:
    call func

In this example, when func2 calls func, func2 is referred to as the caller and func as the callee.

Any function can be either a caller or a callee (or both in different contexts), unless it contains no call instructions. In that latter case, it would be called a "leaf function", and some optimizations may apply.

Answer 2

When some function/subroutine A calls some other function/subroutine B, A is the caller and B is the callee. If B calls C, then B becomes the caller of C and C becomes the callee of B. So, B is both a caller and a callee, depending on how you look at it.

Answer 3

In assembly language function call is not much different from simple jump (a.k.a. goto). The only difference is that function parameters are stored in some well known place before doing jump, so function could read and use them. One of the parameters is usually an address where function should jump after it completed its work, so called "return address". Before jumping to this address, function may store return value in some well known place.

So caller is a piece of code that stores parameters is well known place, invokes jump to the beginning of the function and then, after function will jump back to it, reads returned value from well known place.

Callee is a function itself, i.e. piece of code, the caller jumps to, which reads parameters from well known place, performs some work based on them, stores return value in well known place and jumps back to the caller using return address provided.