'printf' function and 'proc' directive behaving weird in ml64(masm x64)?

Question

;this file -> test1.asm

includelib msvcrt.lib
includelib legacy_stdio_definitions.lib

extern _CRT_INIT: proc     ;custom entrypoint needs c-runtime to be initialised by the programmer.
extern printf: proc
extern ExitProcess: proc

.data
    msg db "hello world!", 13, 10, 0

.code
fun proc 
    sub rsp, 32          ;for shadow space
    call _CRT_INIT       ;called _CRT_INIT because I have my own entry point named "fun"
    lea rcx,msg
    call printf
    add rsp, 32
    call ExitProcess
fun endp
end

commandline for assembler: ml64 /nologo /c test1.asm

commandline for linker: link /entry:fun /subsystem:console test1.obj

This standalone assembly program seems to be working fine. But doing some changes(mentioned below) make the program crash.

1st change -> according to this link both libcmt.lib and msvcrt.lib statically links the native CRT startup (ie both can be used to call _CRT_INIT) into my code. The difference is msvcrt.lib is used with dll. As I don't have any dll I considered using libcmt.lib in place of msvcrt.lib but then the program crashes. Considering there is only printf function associated, that concludes printf function breaks the program. But why?

2nd change -> Now I tried calling this fun() function from a simple .C file. For that I made the necessary changes.

;this file -> test1.asm

;not including any libraries. linking the .c file make the libraries link too because that is in my libpath environment variable

;extern _CRT_INIT: proc    ;no need of _CRT_INIT because now mainCRTStartup() will be the entrypoint and .c file will take care of initializing the c-runtime
extern printf: proc
;extern ExitProcess: proc  ;no need of exitprocess. Instead i'm using ret instruction because I'm calling fun() from .c file.

.data
    msg db "hello world!", 13, 10, 0

.code
fun proc c               ;even if I replace 'c' with something like "abed" the program works. but if I dont give anything after proc then the program crashes.
    sub rsp, 32          ;for shadow space
    lea rcx,msg
    call printf
    add rsp, 32
    ret
fun endp
end

//this file -> test2.c
#include <stdio.h>
#include <conio.h>
void fun();    //in C no need of extern keyword. extern is needed in c++
int main()
{
   fun();
   getch();
}

commandline for assembler: ml64 /nologo /c test1.asm

commandline for compiler: cl /nologo /c test2.c

commandline for linker: link test1.obj test2.obj

In the previous working example (ie. with msvcrt.lib) I don't have to give any suffixes after 'proc' directive. But in this case when I am calling 'fun' as a function from my C file I need to give anything like 'a'/'b'/combination('abcd') after 'proc' directive and only then the program works. If I don't give anything after 'proc' the program crashes. According to official MSDN documentation proc directive also accept language type. Even so wrong/random 'language type name' ie. any word seem to work. But how?

I tried many things like changing the libraries, using different version of printf (like vfprintf) and tried assigning more shadow space also googled much but unable to find any answer.

Note you should subtract 40 not 32 from `rsp` if you do not use a stack frame to maintain alignment. — Jester, Jan 01 '23 at 17:17
40h+8h (8 for the return address that is on the stack because of the call to `fun`) is divisible by 16. Just before `fun` was called the stack was 16 byte aligned. The call itself misaligns the stack by 8. because an 8 byte return address is put on the stack. So you have to account for the misaligned stack when computing adjustments to RSP to maintain alignment if you call a function inside `fun` — Michael Petch, Jan 01 '23 at 17:34
I called fun( ) from my C code that implies when the compiler compiles the .c file it automatically makes the stack allignment for me. — Aniket Bose, Jan 01 '23 at 17:42
https://sonictk.github.io/asm_tutorial/#windows:thewindowtothehardware/themicrosoftx64callingconvention/theshadowspace this guide shows minimum shadow space required is 32 bytes and aslo stack need to be aligned in 16 bytes — Aniket Bose, Jan 01 '23 at 17:43
The stack was aligned in `main` just before it did a `call fun`. The problem is that the `call` instruction misaligns the stack by 8 when it pushes the return address on the stack, and you have to fix that if you call a function like `printf` from `fun`. In `fun` subtracting 8 from RSP gets you back to 16 byte aligned. Subtracting an additional 32 (32 is divisible by 16 so stack alignment is maintained) is enough for the shadow (home space). 32+8=40. — Michael Petch, Jan 01 '23 at 17:47
so according to you that means I need to also take care of function calls in my `main`? But that does not make sense at all. As far as I know for any function called in `main` the stack is alligned by the compiler — Aniket Bose, Jan 01 '23 at 17:51
No, not in `main`. In your `fun`. You already have a return address on the stack at function entry. If you use a standard stack frame you have a `push rbp` which pairs with that to maintain alignment. If you do not use a stack frame you need to account for those extra 8 bytes which is why you should subtract 40. Note this might not have anything to do with your crash but should be fixed nevertheless. — Jester, Jan 01 '23 at 18:00
No you don't need to change `main`. You need to make sure `fun` realigns the stack back to 16 bytes because the `call` instruction used from `main` misaligns it. The ABI says calling an ABI compliant function you need to make sure the stack is 16- byte aligned right before the `call`. What often isn't clear is that the call to a function misaligns it and a function has to account for that. — Michael Petch, Jan 01 '23 at 18:01
If you are also wondering why using `fun proc c` works, it is because adding `c` to the `PROC` will generate prologue code that happens to push RBP on the stack. That `PUSH` subtracts 8 from RSP and happens to realign things. You then subtract an additional 32 so things remain aligned. However when you use `fun proc` without anything after it, the prologue code that includes pushing RBP on the stack isn't done so you have to adjust RSP yourself to account for that. If you review the code actually generated when using `fun proc c` versus `fun proc` you'll see it is different. — Michael Petch, Jan 01 '23 at 18:03
I suspect your `printf` requires a 16 byte aligned stack or it won't work. What version of the Microsoft C compiler (CL) are you using? — Michael Petch, Jan 01 '23 at 18:13
ok I got that. Thanks it helped. Basically I was confused reading the Microsoft doc about stack. It's completely messed up. Please suggest me a good source for understanding stack allignment with function calls @MichaelPetch — Aniket Bose, Jan 01 '23 at 18:20
No, you don't need to delete it. We can mark it as a duplicate of some others. I found one. Jester discusses the alignment issue in this answer: https://stackoverflow.com/a/16422208/3857942 . I'd reopen this question if someone wants to answer the part about why `fun proc c` and `fun proc` differ and how it relates to stack alignment. I haven't actually found an answer already on SO about that (it may exist). Ultimately the issue here is stack alignment. — Michael Petch, Jan 01 '23 at 18:29
This answers the 2'nd part of the problem. what about the 1st part? ( ie why libcmt don't work? ) @MichaelPetch — Aniket Bose, Jan 01 '23 at 18:41
Use a disassembler to see what machine code MASM actually created from your source. Having it magically emit extra instructions that affect stack alignment seems very confusing when you're trying to learn about this stuff; I'd recommend a simpler assembler like NASM. Or I guess just avoiding the magic features like `fun proc c`, but to avoid MASM inventing instructions you didn't write, you'd have to know about all its features in order to not use them. — Peter Cordes, Jan 02 '23 at 03:20

'printf' function and 'proc' directive behaving weird in ml64(masm x64)?

0 Answers0