3

I need to write a C function that will be invoked from assembly code in linux kernel.

What special issues should be taken into account?

I have some in mind, but can anybody provide more details:

(1) Calling convention

Make sure the caller in assembly and the callee in c shake hands well. But which calling convention should I use? How can I declare the c function and declare it in assembly code?

(2) Protect registers

Some registers should be saved in the assembly before the invoke. I roughly remember they are eax, ecx and edx. But I don't have to save them unless I need to refer to the old value before the invoke to the C function, right?

What are other issues?

Infinite
  • 3,198
  • 4
  • 27
  • 36
  • 1
    "I need to write a C function that will be invoked from assembly code in linux kernel." - what's stopping you? – Mitch Wheat Apr 08 '12 at 08:00
  • @MitchWheat Specifically, I don't know the exact usage of "asmlinkage" when declaring the c function. Do I have to use it? Are there any other choices? What is the implication? – Infinite Apr 08 '12 at 08:02
  • 1
    First - try not to do it. Linux uses assembly in a few places, and they're all tricky to mess with. I think the issues depend on what assembly code it is. Running in the very early stages of an interrupt handler, or in the early boot stages (two examples of places where C is used) are two different things. – ugoren Apr 08 '12 at 09:34
  • `asmlinkage` means that all the arguments are passed through the stack. And IIRC `asmlinkage` does nothing with 64-bit code. – Ilya Matveychikov Apr 08 '12 at 09:59
  • Make a simple library with a function in it. Make a hello program that calls the library. Most toy compilers come with a test library and hello program. Disassemble them with a debugger like gdb and note how the arguments are passed. – hellork Apr 08 '12 at 11:04
  • @ugoren Good point. I will need to add a call before the real system call handling function is invoked. – Infinite Apr 08 '12 at 14:50
  • @IlyaMatvejchikov Yes. I know those. After some exploring, I got some conclusions here http://stackoverflow.com/questions/10060168/is-asmlinkage-required-for-a-c-function-to-be-called-from-assembly/10063603#10063603 But I still need a more clear guideline that helps me write such code. – Infinite Apr 08 '12 at 14:52

1 Answers1

2

As your questing tagged with linux-kernel look at the following kernel header: arch/x86/include/asm/calling.h.

   3 x86 function call convention, 64-bit:
   4 -------------------------------------
   5  arguments           |  callee-saved      | extra caller-saved | return
   6 [callee-clobbered]   |                    | [callee-clobbered] |
   7 ---------------------------------------------------------------------------
   8 rdi rsi rdx rcx r8-9 | rbx rbp [*] r12-15 | r10-11             | rax, rdx [**]
   9
  10 ( rsp is obviously invariant across normal function calls. (gcc can 'merge'
  11   functions when it sees tail-call optimization possibilities) rflags is
  12   clobbered. Leftover arguments are passed over the stack frame.)
  13
  14 [*]  In the frame-pointers case rbp is fixed to the stack frame.
  15
  16 [**] for struct return values wider than 64 bits the return convention is a
  17      bit more complex: up to 128 bits width we return small structures
  18      straight in rax, rdx. For structures larger than that (3 words or
  19      larger) the caller puts a pointer to an on-stack return struct
  20      [allocated in the caller's stack frame] into the first argument - i.e.
  21      into rdi. All other arguments shift up by one in this case.
  22      Fortunately this case is rare in the kernel.
  23
  24 For 32-bit we have the following conventions - kernel is built with
  25 -mregparm=3 and -freg-struct-return:
  26
  27 x86 function calling convention, 32-bit:
  28 ----------------------------------------
  29  arguments         | callee-saved        | extra caller-saved | return
  30 [callee-clobbered] |                     | [callee-clobbered] |
  31 -------------------------------------------------------------------------
  32 eax edx ecx        | ebx edi esi ebp [*] | <none>             | eax, edx [**]
  33
  34 ( here too esp is obviously invariant across normal function calls. eflags
  35   is clobbered. Leftover arguments are passed over the stack frame. )
  36
  37 [*]  In the frame-pointers case ebp is fixed to the stack frame.
  38
  39 [**] We build with -freg-struct-return, which on 32-bit means similar
  40      semantics as on 64-bit: edx can be used for a second return value
  41      (i.e. covering integer and structure sizes up to 64 bits) - after that
  42      it gets more complex and more expensive: 3-word or larger struct returns
  43      get done in the caller's frame and the pointer to the return struct goes
  44      into regparm0, i.e. eax - the other arguments shift up and the
  45      function's register parameters degenerate to regparm=2 in essence.
Ilya Matveychikov
  • 3,936
  • 2
  • 27
  • 42
  • Great! That answers a lot of my confusion. Can you tell me how you found such nice document? A silly question. But seriously, I searched hours, but did not find such a document. BTW: can you suggest some books/links that can guide the kernel programming, especially the low-level fashion? Thanks a lot! – Infinite Apr 08 '12 at 15:33
  • Well, I just know where to look :) Speaking about the books you can search for similar questions here. Besides that you can always look at the Linux kernel sources itself. Also, questions are welcome. – Ilya Matveychikov Apr 08 '12 at 16:15
  • The main reference for the x86_64 calling convention is http://www.x86-64.org/documentation/abi.pdf ("the ABI" - _Application Binary Interface_). It lists calling conventions and compound data (`struct`) as well a 'primitive' type layout/size/alignment constraints. The conventions used by the Linux kernel in 64bit x86 are extremely close to this. In 32bit, they're quite different from the userspace ABI (search the web for _gabi.pdf_ if you're curious), though, due to the use of `gcc -mregparm' - that's where the kernel sources come in handy. – FrankH. May 31 '12 at 17:13