0

I have a function in an unmanaged Win32 DLL that takes a variable number of arguments and therefore needs to be __cdecl rather than __stdcall or any other calling convention. At the moment I'm linking explicitly to the DLL (which is to say, I'm using LoadLibrary and GetProcAddress rather than linking to the DLL's .lib file).

I can call the function just fine, but since it's cdecl I need to add the following inline assembly instruction after each call:

retVal = addVecs(v1, v2, v3, v4);
__asm add esp, 64 ;Sizeof VECTOR struct is 16 bytes
printf("The sum of all these vectors is:\n\tMagnitude: %f\n\tDirection (radians): %f\n\n", retVal.mag, retVal.dir);

If I do not include the inline assembly then the program crashes after the call to addVecs.

Is there any way I can write either the EXE or the DLL such that the inline assembly instruction is not needed? For example, printf is also a __cdecl function yet I do not need to write inline assembly to ensure the stack is cleaned up after every call I make to it.

Govind Parmar
  • 20,656
  • 7
  • 53
  • 85
  • 2
    When you store the function pointer after calling `GetProcAddress`, do you store it in a pointer that includes `__cdecl`? – paddy May 28 '15 at 03:04
  • Isn't __cdecl usually the default calling convention in Win32? (At least for code compiled with MSVC, and even though the Win32 API itself doesn't use it) – user253751 May 28 '15 at 03:42
  • @immibis You can specify the default calling convention as a compiler option. It's not something you should rely on. Especially when loading DLLs, you should always be explicit. – paddy May 28 '15 at 03:46

1 Answers1

1

You need to make sure the calling convention of addVecs is correct. It pays to be explicit about this, and not rely on your compiler's defaults.

typedef VECTOR (__cdecl *addVecs_ptr)( VECTOR, VECTOR, VECTOR, VECTOR );

static addVecs_ptr addVecs = NULL;

If you've used a typedef like this, it makes the cast easy later on when you're loading the address:

addVecs = (addVecs_ptr) GetProcAddress( hSomeDllHandle, "addVecs" );
paddy
  • 60,864
  • 6
  • 61
  • 103
  • Helps a lot! I've gotten used to typing in the standard CALLBACK* macro when using GetProcAddress that I forgot that that's a #define for __stdcall – Govind Parmar May 28 '15 at 04:38