1

I'm calling a function from C that I define using NASM 2.13.3.1. I'm developing in Visual C++ Professional 2017 15.9.21.

Here is the C code:

#include <string.h>

extern _begin_asm_func(int, const char*, void*);

struct MyStruct
{
    int64_t a;
    int64_t b;
};

int main(int argc, char ** argv)
{
    struct MyStruct x;
    x.a = 1;
    x.b = 2;

    char str[] = "Hi";
    int len = strlen(str);
    
    _begin_asm_func(len, str, (void *)&x);

    return 0;
}

NASM Code:

global _begin_asm_func

SECTION .data

SECTION .text
_begin_asm_func:
        ; align stack pointer to 16-byte boundary because return address (8 bytes) pushed by call \
        PUSH rbx ;
        PUSH r12 ;
        PUSH r13 ;
        PUSH r14 ;
        PUSH r15 ;
        ; five pushes to save non-volitile registers for use later should also align stack

        MOV r12, rcx ; store length in r12
        MOV r13, rdx ; store string pointer in r13
        MOV r14, r8  ; store pointer to data structure in r14
        MOV r15, [r8] ; store first field value
        
        ; useful stuff will go here later...

        POP r15 ;
        POP r14 ;
        POP r13 ;
        POP r12 ;
        POP rbx ;
        RET ; access violation happens here

And here is the debugger output:

global _begin_asm_func

SECTION .data

SECTION .text

_begin_synergy_asm:
00007FF7E37718A0  push        r12
00007FF7E37718A2  push        r13
00007FF7E37718A4  push        r14
00007FF7E37718A6  push        r15
00007FF7E37718A8  mov         r12, rcx
00007FF7E37718AB  mov         r13, rdx
00007FF7E37718AE  mov         r14, r8
00007FF7E37718B1  mov         r15, qword ptr[r8]
00007FF7E37718B4  pop         r15
00007FF7E37718B6  pop         r14
00007FF7E37718B8  pop         r13
00007FF7E37718BA  pop         r12
00007FF7E37718BC  pop         rbx
00007FF7E37718BD  ret

The seg fault / access violation happens on return. Two things immediately jumped out at me. First, the first push to rbx is missing, and second, the address of the second push is on a 4-byte boundary. So it seems to me that the instructions are being mis-aligned by nasm, so the call is landing on the second instruction instead of the first. That mis-aligns the stack so that the number of pops don't match the mumber of pushes, so that ret gets the wrong value for the return address.

Assuming alignment is the actual problem, how do I tell nasm to align the label for the function? I know the align directive is only for aligning data.

If that isn't the issue, then what is it and how can I avoid it?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
hatch22
  • 797
  • 6
  • 18
  • Minor nitpick: `align` is also used for aligning code in NASM, which is why it defaults to emit 90h (`nop` opcode) bytes to do the aligning. That means you can put `align` before a code label (like for data) or even after the label. However, this is unlikely to help here. – ecm Feb 12 '21 at 16:55
  • This usually happens if the preceding comment line is accidentally continued to the `push rbx`. – Jester Feb 12 '21 at 17:29
  • @ecm I tried align just in case and it did not make a difference, as you predicted. – hatch22 Feb 12 '21 at 17:34
  • @Jester Can you give me an example of what you mean? – hatch22 Feb 12 '21 at 17:35
  • 1
    An example would be an accidental backslash at the end of the preceding line. Try putting a blank line before the `push rbx`. – Jester Feb 12 '21 at 17:40
  • 2
    And so there is an accidental backslash! Thank you for pointing me in that direction. I feel foolish for not seeing that, but it explains everything nicely. – hatch22 Feb 12 '21 at 17:51
  • Relevant: https://bugzilla.nasm.us/show_bug.cgi?id=3392663 – ecm Feb 12 '21 at 17:54
  • Not a [mcve], code in the question doesn't contain a \ and copy-pasting into a .asm doesn't repro the problem. But yes, you did correctly diagnose the problem from looking at disassembly. No need to bring MSVC into it. – Peter Cordes Mar 04 '21 at 18:02
  • Added the backslash in my original code. Note sure why it didn't copy over. – hatch22 Mar 05 '21 at 00:31

0 Answers0