Parameters aren’t passed to unmanaged DLL on x86 when building with .NET Native

Question

I’m building a Windows 10 universal application (phone + tablets) + libraries. In the solution I have C++ dll project that builds unmanaged my.dll that’s called from C#. The DLL has export like this:

// === C++ ===
typedef struct { int f1; uint32_t f2; } R;
// A and B are also structures.
MY_EXPORT R the_function( A *a, const B *b, const uint8_t *c );

// === C# ===
[DllImport( "my.dll", ExactSpelling = true, CallingConvention = CallingConvention.Cdecl )]
extern static R the_function(A a, B b, byte[] c);

[StructLayout( LayoutKind.Sequential )]
internal struct R
{
    public int f1;  // Actually a enum but it shouldn’t matter.
    public uint f2_id;
} 

internal struct A
{
    IntPtr nativePtr;
}

internal struct B
{
    IntPtr nativePtr;
}

The test app works on ARM and X64 platforms. It works on X86 if "Compile with .NET Native tool chain" is unchecked.

The unmanaged DLL crashes on X86 if "Compile with .NET Native tool chain" is checked, saying access violation. I can reproduce in both Debug and Release builds.

When using the debugger, I see there’s an error in how the arguments are passed. On the C# side, in some compiler-generated C# code there’s a line like this:

unsafe___value = global::McgInterop.my_PInvokes.the_function( a, b, unsafe_c );

In the debugger, I confirm the arguments are OK.

On C++ side, the values are wrong. The b's value is what was passed in a, the c's value is what was passed in b.

I tried to create a minimalistic example but failed, it works OK. my.dll exports 100+ exported __cdecl method, it's a large cross-platform C++ SDK I'm working on to bring to Windows 10 platform, looks like the rest of methods work OK.

Any ideas what's happening here? Or at least how do I isolate the issue? Thanks in advance.

Update: OK here's a minimal repro.

Unmanaged code:

typedef struct
{
    int f1;
    DWORD f2;
} R;

R __cdecl nativeBug( int a, int b )
{
    CStringA str;
    str.Format( "Unmanaged DLL: a = %i, b = %i\n", a, b );
    ::OutputDebugStringA( str );
    R res
    {
        11, 12
    };
    return res;
}

C# store app:

[StructLayout( LayoutKind.Sequential )]
struct R
{
    public int f1;
    public uint f2;
}

[DllImport( "NativeBugDll.dll", ExactSpelling = true, CallingConvention = CallingConvention.Cdecl )]
extern static R nativeBug( int a, int b );

private void Page_Loaded( object sender, RoutedEventArgs e )
{
    App.Current.UnhandledException += app_UnhandledException;
    R r = nativeBug( 1, 2 );
    Debug.WriteLine( "Result: f1={0}, f2={1}", r.f1, r.f2 );
}

private void app_UnhandledException( object sender, UnhandledExceptionEventArgs e )
{
    Debug.WriteLine( "Unhandled exception: " + e.Message );
}

Debug output without .NET Native is fine:

Unmanaged DLL: a = 1, b = 2
Result: f1=11, f2=12

And here's debug output with .NET Native build:

Unmanaged DLL: a = 91484652, b = 1
Unhandled exception: Object reference not set to an instance of an object.
STATUS_STACK_BUFFER_OVERRUN encountered

Then visual studio hangs completely.

The X64 build works fine even with .NET Native.

Answer 1

Yikes! Looks like there may be a bug in .NET Native. I've asked someone here at Microsoft to take a look. If you want to get hooked up with us internally feel free to mail us at dotnetnative@microsoft.com.

I'll update this as we know more.

EDIT: So there is definitely a real bug if a native function returns structs like this. The optimizer has ended up in a state where it pushes one extra argument to the stack after the two parameters and that’s what causing the bug.

I've opened a bug and we'll get this fixed for Update 2 of VS.

C#:

[StructLayout(LayoutKind.Sequential)]
struct R
{
   public int f1;
   public int f2;
    }

[DllImport("DllImport_NativeDll.dll")]
extern static R nativeBug(int a, int b);

public static void Run()
{
        R r = nativeBug(1, 2);
}

Native:

typedef struct
{
    int f1;
    int f2;
} R;

extern "C" __declspec(dllexport) R nativeBug(int a, int b)
{
    R res
    {
        11, 12
    };
    return res;
}

Code generated:

00f1766b 8b55fc          mov     edx,dword ptr [ebp-4]
00f1766e 52              push    edx
00f1766f 8b45f8          mov     eax,dword ptr [ebp-8]
00f17672 50              push    eax
00f17673 8d4ddc          lea     ecx,[ebp-24h]
00f17676 51              push    ecx <-- Bonus and unfortunate push
00f176ab ff1524b4d200    call    dword ptr [PInvokeAndCom!_imp__nativeBug (00d2b424)]

Answer 2

MY_EXPORT R the_function( A *a, const B *b, const uint8_t *c );

The first two arguments contain the addresses of the structs.

extern static R the_function(A a, B b, byte[] c);

Here you are passing the structs by value. Given the code in the question, that's the only difference that I can see. To pass addresses of the structs change the C# to:

extern static R the_function(ref A a, ref B b, byte[] c);