1

Context: I am porting some old VB.NET code from x86 to AnyCPU platform.

Problem statement: In a class instance, I noticed a call to the kernel32 CreatePipe function would unexpectedly change, when run as a 64-bit process, the value of a totally unrelated boolean class variable from True to False. No exception thrown nor anything else, just that value change I see in the watch when that call to CreatePipe occurs.

What I did to fix this issue: Based on the pinvoke website (https://www.pinvoke.net/default.aspx/kernel32.createpipe), and then from one of the comments below, I changed the CreatePipe signature in my class from:

Private Declare Function CreatePipe Lib "kernel32" (ByRef phReadPipe As Integer,
                                                    ByRef phWritePipe As Integer,
                                                    ByRef lpPipeAttributes As SECURITY_ATTRIBUTES,
                                                    ByVal nSize As Integer) As Integer

To:

Private Declare Function CreatePipe Lib "kernel32" (ByRef phReadPipe As IntPtr,
                                                    ByRef phWritePipe As IntPtr,
                                                    ByRef lpPipeAttributes As SECURITY_ATTRIBUTES,
                                                    ByVal nSize As Integer) As Integer

And it works now, I mean, the boolean value is not unexpectedly changed anymore and the rest of the code behaves as intended.

Questions: I am not extremely satisfied by that though, and I won't be until I know what caused this behavior in a 64-bit process. Wrong memory access causing the flag value to change? Why this one variable in particular? Is it due to the incorrect length specification on the variables which may have caused the call to overwrite the next few bytes of data (this may be where the boolean variable lives)? This is just speculation though...

More generally, is there any guidelines out there to port .NET code from an x86 platform to AnyCPU, when there is one or many calls to the Windows API?

CTZStef
  • 1,675
  • 2
  • 18
  • 47
  • 1
    I cannot answer the question about the Boolean class variable (I suggest that you create a [mcve], if possible), but I can answer your question about the generic guideline: Every value which is a `pointer` (e.g. `phReadPipe` and `phWritePipe`) should be declared as `IntPtr` rather than `Integer`/`Long`. This will (a) ensure that it has the correct size for each bitness and (b) prevent you from treating it like a regular number. – Heinzi Apr 22 '20 at 11:12
  • 1
    BTW, the return type, `BOOL` is an alias for `int`, which is always 32 bit on Windows (even in 64 bit processes), likewise `DWORD` is always 32 bit as well. Thus, the correct signature should be `(IntPtr, IntPtr, SECURITY_ATTRIBUTES, Integer) As Integer` (although, personally, I prefer the more descriptive `Int32` over `Integer`). – Heinzi Apr 22 '20 at 11:21
  • 1
    Here's a link with decriptions of all those Windows API data types. "Converting" them to VB.NET manually is usually more reliable than copy & pasting it from some community website such as pinvoke.net: https://learn.microsoft.com/en-us/windows/win32/winprog/windows-data-types – Heinzi Apr 22 '20 at 11:23
  • @Heinzi thank you. It is not possible to create a minimal reproducible example here I think, because of the nature of the problem, I guess I would have to share the whole project for this issue to happen, although it would be interesting to try. Thanks again for all the info. – CTZStef Apr 22 '20 at 11:25
  • (Obviously, `ByRef` already adds one level over "pointer-ness", so that should be kept in mind as well.) – Heinzi Apr 22 '20 at 11:25
  • You can also create a minimal example by copying your project and *removing* stuff until only the problem remains. That's usually a lot of work, though. – Heinzi Apr 22 '20 at 11:26
  • I know what you mean, we recently did the same with a large legacy VBA project (modifying all Windows API calls to be 64-bit compatible). – Heinzi Apr 22 '20 at 11:33

0 Answers0