Correct way to marshall uchar[] from native dll to byte[] in c#

Question

I'm trying to marshall some data that my native dll allocated via CoTaskMemAlloc into my c# application and wondering if the way I'm doing it is just plain wrong or I'm missing some sublte decorating of the method c# side.

Currently I have c++ side.

extern "C" __declspec(dllexport) bool __stdcall CompressData(  unsigned char* pInputData, unsigned int inSize, unsigned char*& pOutputBuffer, unsigned int& uOutputSize)
{ ...
    pOutputBuffer = static_cast<unsigned char*>(CoTaskMemAlloc(60000));
    uOutputSize = 60000;

And on the C# side.

    private const string dllName = "TestDll.dll";

    [System.Security.SuppressUnmanagedCodeSecurity]
    [DllImport(dllName)]
    public static extern bool CompressData(byte[] inputData, uint inputSize, out byte[] outputData, out uint outputSize );
    ...
    byte[] outputData;
    uint outputSize;
    bool ret = CompressData(packEntry.uncompressedData, (uint)packEntry.uncompressedData.Length, out outputData, out outputSize);

here outputSize is 60000 as expected, but outputData has a size of 1, and when I memset the buffer c++ side, it seems to only copy across 1 byte, so is this just wrong and I need to marshall the data outside the call using an IntPtr + outputSize, or is there something sublte I'm missing to get working what I have already?

Thanks.

The current behaviour makes sense: as there is nothing to indicate that `outputSize` is related in any way at all to `outputData`, so `outputData` is treated as a pointer to a single `unsigned char`. Unfortunately, I cannot help you with the correct way to write this, only with an explanation of the current behaviour. — , Oct 29 '12 at 14:37
Yes, that does indeed make sense, the outputSize is for my benefit in testing and obviously if the byte[] knew what it should be would be obsolete. :) Thanks anyway. — Niksan, Oct 29 '12 at 14:53

score 5 · Answer 1 · answered Oct 29 '12 at 14:48

There are two things.

First, the P/Invoke layer does not handle reference parameters in C++, it can only work with pointers. The last two parameters (pOutputBuffer and uOutputSize) in particular are not guaranteed to marshal correctly.

I suggest you change your C++ method declaration to (or create a wrapper of the form):

extern "C" __declspec(dllexport) bool __stdcall CompressData(  
    unsigned char* pInputData, unsigned int inSize, 
    unsigned char** pOutputBuffer, unsigned int* uOutputSize)

That said, the second problem comes from the fact that the P/Invoke layer also doesn't know how to marshal back "raw" arrays (as opposed to say, a SAFEARRAY in COM that knows about it's size) that are allocated in unmanaged code.

This means that on the .NET side, you have to marshal the pointer that is created back, and then marshal the elements in the array manually (as well as dispose of it, if that's your responsibility, which it looks like it is).

Your .NET declaration would look like this:

[System.Security.SuppressUnmanagedCodeSecurity]
[DllImport(dllName)]
public static extern bool CompressData(byte[] inputData, uint inputSize, 
    ref IntPtr outputData, ref uint outputSize);

Once you have the outputData as an IntPtr (this will point to the unmanaged memory), you can convert into a byte array by calling the Copy method on the Marshal class like so:

var bytes = new byte[(int) outputSize];

// Copy.
Marshal.Copy(outputData, bytes, 0, (int) outputSize);

Note that if the responsibility is yours to free the memory, you can call the FreeCoTaskMem method, like so:

Marshal.FreeCoTaskMem(outputData);

Of course, you can wrap this up into something nicer, like so:

static byte[] CompressData(byte[] input, int size)
{
    // The output buffer.
    IntPtr output = IntPtr.Zero;

    // Wrap in a try/finally, to make sure unmanaged array
    // is cleaned up.
    try
    {
        // Length.
        uint length = 0;

        // Make the call.
        CompressData(input, size, ref output, ref length);

        // Allocate the bytes.
        var bytes = new byte[(int) length)];

        // Copy.
        Marshal.Copy(output, bytes, 0, bytes.Length);

        // Return the byte array.
        return bytes;
    }
    finally
    {
        // If the pointer is not zero, free.
        if (output != IntPtr.Zero) Marshal.FreeCoTaskMem(output);
    }
}

Thanks for that, I wasn't aware of SAFEARRAY which I'd sooner use to be honest if that takes the burden of the caller clearing up the memory (as opposed to the invoker / GC dealing with it, which I assume it does using SAFEARRAY?) If not, I'll go the IntPtr/Copy route, I was just looking for way to avoid it. Upvote pending. :) — Niksan, Oct 29 '12 at 15:03
@Niksan `SAFEARRAY` will create issues on the C++ side, and the code above isn't that bad if you wrap it like I did in the end (you only have to write that code once). In the end, whatever is easier for you. — casperOne, Oct 29 '12 at 15:05
Well, the easiest for me would be a copy, but knowing about SAFEARRAY wouldn't go amiss in my arsenal, what problems would be caused on the c++ side in regards to SAFEARRAY? I'm currently dealing with new[]/delete[] combo in my dll and do a final CoTaskMemAlloc as a final push to .NET due to potential reallocs, I'd effectively do the same for a SAFEARRAY, but if it's more trouble than it's worth I'll skip. Thanks again. — Niksan, Oct 29 '12 at 15:15
@casperOne: hi, how do you malloc this `unsigned char** pOutputBuffer` in c++ code? I want to copy some data into this buffer in c++ code — Agung Pratama, Mar 12 '13 at 10:56
If you're using unmanaged code, then you can use any mechanism you want. In C++, you would use `new` or `malloc` most likely. However, if you have to release the memory in *managed* code, then you'll need to use a mechanism which can be allocated and freed from managed and unmanaged code, such as `CoTaskMemAlloc` or `LocalAlloc`. — casperOne, Mar 12 '13 at 11:01

Hans Passant · Answer 2 · 2012-10-30T09:50:31.490

The pinvoke marshaller cannot guess how large the returned byte[] might be. Raw pointers to memory in C++ do not have a discoverable size of the pointed-to memory block. Which is why you added the uOutputSize argument. Good for the client program but not quite good enough for the pinvoke marshaller. You have to help and apply the [MarshalAs] attribute to pOutputBuffer, specifying the SizeParamIndex property.

Do note that the array is getting copied by the marshaller. That's not so desirable, you can avoid it by allowing the client code to pass an array. The marshaller will pin it and pass the pointer to the managed array. The only trouble is that the client code will have no decent way to guess how large to make the array. The typical solution is to allow the client to call it twice, first with uOutputSize = 0, the function returns the required array size. Which would make the C++ function look like this:

extern "C" __declspec(dllexport) 
int __stdcall CompressData(
     const unsigned char* pInputData, unsigned int inSize, 
     [Out]unsigned char* pOutputBuffer, unsigned int uOutputSize)

Correct way to marshall uchar[] from native dll to byte[] in c#

2 Answers2

Linked