2

I have a Windows Forms application running under .NET 4.0. This application imports a DLL which is available for:

  • 32 bit
  • 64 bit

Here is my code snippet:

[DllImport("my64Bit.dll"), EntryPoint="GetLastErrorText"]
private static extern string GetLastErrorText();

// Do some stuff...

string message = GetLastErrorText();

When calling this function (compiled for x64) the application just crashes. I can't even see any debug message in Visual Studio 2012. The identical code with the 32-bit-DLL (compiled for x86) works fine. The prototype is:

LPCSTR APIENTRY GetLastErrorText()

Unfortunately I don't have any further information about the DLL as it is a third-party product.

Robert Strauch
  • 12,055
  • 24
  • 120
  • 192
  • http://stackoverflow.com/questions/3598226/how-to-map-win32-types-to-c-sharp-types-when-using-p-invoke – NickD Dec 18 '12 at 11:50
  • LPCSTR is char*. Try to add CharSet = CharSet.Ansi to DllImport. – Alex F Dec 18 '12 at 11:58
  • @AlexFarber As far as I know the default CharSet is Ansi. Anyway, the application still crashes. As mentioned, the 32 bit version is running fine. – Robert Strauch Dec 18 '12 at 12:03
  • Why are not using `AnyCPU` and instead trying to target to specific platforms? The problem of course is the 64-bit process is likely trying to load the 32-bit version of the dll. My guess is that `my64Bit.dll` isn't actually compiled to be a 64-bit dll. – Security Hound Dec 18 '12 at 12:35
  • @Ramhound Actually I'm using `AnyCPU`. This way the application works fine on a x86 system. I did a workaround to load the 32-bit-DLL in this case. Of course, maybe the 64-bit-DLL itself is buggy. – Robert Strauch Dec 18 '12 at 12:38
  • 1
    @RobertStrauch - Sounds like you have lots of research to do. – Security Hound Dec 18 '12 at 12:59

2 Answers2

5

The function signature is quite troublesome. Whether your code will crash depends on what operating system you run. Nothing happens on XP, an AccessViolation exception is thrown on Vista and later.

At issue is that C functions returning strings need to typically do so by returning a pointer to a buffer that stores a string. That buffer needs to be allocated from the heap and the caller needs to release that buffer after using the string. The pinvoke marshaller implements that contract, it calls CoTaskMemFree() on the returned string pointer after converting it to a System.String.

That invariably turns out poorly, a C function almost never uses CoTaskMemAlloc() to allocate the buffer. The XP heap manager is very forgiving, it simply ignores bad pointers. Not the later Windows versions, they intentionally generate an exception. A strong enabler for the "Vista sucks" label btw, it took a while for programmers to get their pointer bugs fixed. If you have unmanaged debugging enabled then you'll get a diagnostic from the heap manager which warns that the pointer is invalid. Very nice feature but unmanaged debugging is invariably disabled when you debug managed code.

You can stop the pinvoke marshaller from trying to release the string by declaring the return value as IntPtr. You then have to marshal the string yourself with Marshal.PtrToStringAnsi() or one of its friends.

You still have the problem of having to release the string buffer. There is no way to do this reliably, you cannot call the proper deallocator. The only hope you have is that the C function actually returns a pointer to a string literal, one that's stored in the data segment and should not be released. That might work for a function that returns an error string, provided it doesn't implement anything fancy like localization. The const char* return type is encouraging.

You will need to test this to make sure there is no memory leak from not releasing the string buffer. Easy to do, call this function a billion times in a loop. If you don't get IntPtr.Zero as a return value and the program doesn't otherwise fall over with a out-of-memory exception then you're good. For 64-bit pinvoke you'll need to keep an eye on the test program's memory consumption.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
2

Found it. The native function returns LPCSTR, i.e. the C# function cannot return a string. Instead an IntPtr must be returned like this:

[DllImport("my64Bit.dll"), EntryPoint="GetLastErrorText"]
private static extern IntPtr GetLastErrorText();

// Do some stuff...

IntPtr ptr = GetLastErrorText();
string s = Marshal.PtrToStringAnsi(ptr);
Robert Strauch
  • 12,055
  • 24
  • 120
  • 192
  • Interesting, why standard marshalling with string doesn't work in this case. Anyway, working with IntPtr and Marshal is always better for a programmer who knows C/C++. – Alex F Dec 19 '12 at 18:12