PInvoke signature does not match the unmanaged target signature

Question

I'm trying to create a wrapper C# file for modbusm.dll (win-tech.com/html/mbusocx.htm) ,I'm using dumpbin output for this.

Dump of file modbusm.dll

File Type: DLL

Section contains the following exports for modbusm.dll

00000000 characteristics
41128817 time date stamp Fri Aug 06 00:48:47 2004
    0.00 version
       1 ordinal base
      27 number of functions
      27 number of names

ordinal hint RVA      name

      1    0 000085BA _AbortTheCall@4
      2    1 00003441 _CloseConnection@4
      3    2 000033A7 _ConnectASCII@12
      4    3 000033E1 _ConnectDanielsASCII@12
      5    4 000033C4 _ConnectDanielsRTU@12
      6    5 0000338A _ConnectRTU@12
      7    6 00001019 _ConnectTCP2@12
      8    7 00001000 _ConnectTCP@8
      9    8 0000829A _DialCall@8
     10    9 00003376 _EnableConnectionCallback@4
     11    A 00003342 _EnableModbusCallback@8
     12    B 00008123 _GetCallState@8
     13    C 00007FD2 _GetLineDeviceName@12
     14    D 00003320 _GetPollDelay@0
     15    E 00003339 _Get_Modbus_DLL_Revision@0
     16    F 000033FE _HookRspNotification@16
     17   10 000032ED _InitializeWinSock@0
     18   11 0000277C _MBAPWndProc@16
     19   12 0000393F _MODBUSResponse@16
     20   13 00007EAA _NumberOfLineDevices@0
     21   14 00003521 _PollMODBUS@8
     22   15 000039F2 _ReadDebugData@16
     23   16 00003BA4 _ReadTransparentResponse@16
     24   17 0000332A _SetPollDelay@4
     25   18 00003313 _UnInitializeWinSock@0
     26   19 00003712 _WriteMODBUS@12
     27   1A 00003AB3 _WriteTransparentString@12

Summary

    5000 .data
    2000 .rdata
    2000 .reloc
    1000 .rsrc
    C000 .text

my C# wrapper is

class MbMasterV7
    {
        [DllImport("modbusm.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Unicode, EntryPoint = "_ConnectTCP2@12")]
        public static extern int ConnectModbusTCP(short Port);


        public static string TCPDevice { set; get; }
    }

when I am running the code

MbMasterV7.TCPDevice = "127.0.0.1";  // from  demo version of .ocx file converted using tlbimp.exe
MbMasterV7.ConnectModbusTCP(502);

in visual studio, I am getting exception

A call to PInvoke function 'TestApp!TestApp.MbMasterV7::ConnectModbusTCP' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

I have tried all calling conventions and getting same error. The .Net libraries are available for modbus protocol are not good enough for the new plc type I am working with.

FILE HEADER VALUES
             14C machine (x86)
               5 number of sections
        41128817 time date stamp Fri Aug 06 00:48:47 2004
               0 file pointer to symbol table
               0 number of symbols
              E0 size of optional header
            210E characteristics
                   Executable
                   Line numbers stripped
                   Symbols stripped
                   32 bit word machine
                   DLL

OPTIONAL HEADER VALUES
             10B magic # (PE32)
            6.00 linker version
            C000 size of code
            A000 size of initialized data
               0 size of uninitialized data
            94E4 entry point (100094E4)
            1000 base of code
            D000 base of data
        10000000 image base (10000000 to 10016FFF)
            1000 section alignment
            1000 file alignment
            4.00 operating system version
            0.00 image version
            4.00 subsystem version
               0 Win32 version
           17000 size of image
            1000 size of headers
               0 checksum
               2 subsystem (Windows GUI)
               0 DLL characteristics
          100000 size of stack reserve
            1000 size of stack commit
          100000 size of heap reserve
            1000 size of heap commit
               0 loader flags
              10 number of directories
            DF90 [     357] RVA [size] of Export Directory
            D7C8 [      50] RVA [size] of Import Directory
           14000 [     3E8] RVA [size] of Resource Directory
               0 [       0] RVA [size] of Exception Directory
               0 [       0] RVA [size] of Certificates Directory
           15000 [     BC0] RVA [size] of Base Relocation Directory
               0 [       0] RVA [size] of Debug Directory
               0 [       0] RVA [size] of Architecture Directory
               0 [       0] RVA [size] of Global Pointer Directory
               0 [       0] RVA [size] of Thread Storage Directory
               0 [       0] RVA [size] of Load Configuration Directory
               0 [       0] RVA [size] of Bound Import Directory
            D000 [     198] RVA [size] of Import Address Table Directory
               0 [       0] RVA [size] of Delay Import Directory
               0 [       0] RVA [size] of COM Descriptor Directory
               0 [       0] RVA [size] of Reserved Directory

Answer 1

First thing first, _ConnectTCP2@12 means that 12 bytes are passed as parameters to the function, which means that short (which is 2 bytes long) is obviously incompatible. You need to pass 12 bytes as parameters, presumably as 3 DWORDs.

Let us, for the sake of adventure, actually disassemble the binary to see what's going on there.

So: ConnectTCP@8 is receiving 2 DWORDs as arguments and calling ConnectTCP2 with 0x1F6 as the second parameter (which is actually a short). Also, the calling convention is stdcall.

That's enough information for us to figure out how to call the function:

    [DllImport("modbusm.dll", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Ansi, EntryPoint = "_ConnectTCP2@12")]
    public static extern int ConnectModbusTCP(int a, short Port, int b);

Will work, but will throw

 "Unhandled Exception: System.AccessViolationException: 
    Attempted to read or write protected memory.
    This is often an indication that other memory is corrupt."

This is because the second integer (that I called b) is actually a pointer to a structure (which its values I can only guess according to the code). So let us reconstruct the structure. According to the code, the structure is accessed five times as following:

Offset  Type
0x00 -> INT32
0x04 -> INT32
0x08 -> INT32
0x0C -> INT16
0x10 -> INT32

So, by creating the following structure:

struct MbMasterStruct
{
    int a;
    int b;
    int c;
    short d;
    int e;
}

And redefining the function to be:

unsafe class MbMasterV7
{
    [DllImport("modbusm.dll", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Ansi, EntryPoint = "_ConnectTCP2@12")]
    public static extern int ConnectModbusTCP(int a, short Port, MbMasterStruct * b);
}

And calling it as following:

static void Main(string[] args)
{
    var structure = new MbMasterStruct();
    unsafe
    {
        MbMasterV7.ConnectModbusTCP(1, 2, &structure);
    }
}

It actually works, and it doesn't throw. On my computer, it returned 51 (when the structure and parameters were all zero).

It is now up to you to understand each parameters and see how to properly call the function.