Handeling NULL pointer out argument in C# COM

Question

I am working with a COM plugin interface that has the following function definition:

HRESULT foo ( [out, ref] VARIANT* a, [out, ref] VARIANT* b );

When using tlbimp (tlbimp2.exe from codeplex) the .NET library has the following interface function:

int foo ( out object a, out object b );

The problem is that the calling application will make the function call:

VARIANT a;
::InitVariant( &a );

plugin->foo( &a, NULL );

And in C# I implemented the function:

int foo ( out object a, out object b )
{
    a = 1;
    b = 2;

    return 0; // S_OK
}

When all is said and done the application actually gets a E_POINTER return and not S_OK. I assume it is because of the NULL passed the the out parameter.

Is there a way to check if the address pointer is NULL in the C# implementation?

Note: out parameters are not initialized so you cant use the parameters at all.

I've tried implementing the interface as [in, out, ref] to force C# to use ( ref object a, ref object b ) but that did not work either.

Update

Hans is absolutely correct that we should have been calling the function with a NULL ptr if we were declaring it as [out,ref].

erurainon was also correct that we could just use IntPtr to get to the Variant*.

So here is how it was fixed:

int foo ([MarshalAs(UnmanagedType.Struct)]out object a, [MarshalAs(UnmanagedType.Struct)]out object b);

became

int foo ([MarshalAs(UnmanagedType.Struct)]out object a, IntPtr b );

Now we can test for the NULL cases with:

if ( b == IntPtr.Zero )

However since we are dealing with Variant you can't just copy the value to the IntPtr like:

Marshal.StructureToPtr( myValue, b, false );

So following this post, you need to make a struct class:

[StructLayout(LayoutKind.Explicit, Size = 16)]
public struct PropVariant
{
    [FieldOffset(0)]
    public VarEnum variantType;
    [FieldOffset(8)]
    public IntPtr pointerValue;
    [FieldOffset(8)]
    public byte byteValue;
    [FieldOffset(8)]
    public long longValue;
    [FieldOffset(8)]
    public double dateValue;
    [FieldOffset(8)]
    public short boolValue;
}

And the final function looks like this:

int foo( out object a, IntPtr b )
{
    a = 100;

    if ( b != IntPtr.Zero )
    {
        var time = new PropVariant();
        time.dateTime = DateTime.Now.ToOADate();
        time.variantType = VarEnum.VT_DATE;

        Marshal.StructureToPtr( time, b, false );
    }

    return 0; // S_OK
}

Hope this helps someone else in the future

score 3 · Answer 1 · answered Jul 09 '11 at 01:46

3

From the MIDL docs:

The [ref] attribute identifies a reference pointer. It is used simply to represent a level of indirection
....
A reference pointer has the following characteristics:
 - Always points to valid storage; never has the value NULL. A reference pointer can always be dereferenced.

The code is invalid, it passed NULL. A direct violation of the [ref] contract. This is where the bucks stops, you'll have to fix either the MIDL or the native code.

answered Jul 09 '11 at 01:46

Hans Passant

922,412
146
1,693
2,536

Yeah, that is what I figured. Unfortunately the version that calls with NULL has already been shipped. Thankfully it's in a very minor part of the application, but still was just wondering. – maschall Jul 11 '11 at 12:51

score 1 · Accepted Answer · answered Jul 09 '11 at 01:58

1

Hans is right. Either you disallow your API to be called with NULL parameters, or you implement your C# function with IntPtr arguments and check for IntPtr.Zero.

answered Jul 09 '11 at 01:58

John Estropia

17,460
4
46
50

I'll have to give that a try. I'm still hoping I can change the IDL file slightly to allow the application to still work with the plugin properly. – maschall Jul 11 '11 at 12:53
That actually worked just well, the big problem was copying the object to the address when it wasn't NULL/IntPtr.Zero, Thanks – maschall Jul 14 '11 at 13:51

Handeling NULL pointer out argument in C# COM

2 Answers2