1

I have build a C++ CLI wrapper for a communication middleware we use. The wrapper has been in .NET 4.5 and works fine in Windows 7 and Windows Server 2008 R2. But in Windows Server 2012 R2 it crashes with an "Access violation" error in mswsock.dll

The interesting part is that if I build the wrapper and test application to target .NET 4.0 it works. But when I re-target them to .NET 4.5.x it triggers the exception.

I've tried to investigate security, strong naming, hot-fix'ing .NET, but to no use.

What troubles me is that it works on a Server 2012 R2 in .NET 4.0, but not 4.5.x. And that it evidently is when the native code accesses the winsock API that the access violation occurs.

Have anyone experienced the same issue, or similar Server 2012 .NET 4.0 vs 4.5.x wiredness?

Are there any security mechanisms activated in Server 2012 when running .NET 4.5 applications, that aren't active for 4.0 applications. Specifically regarding CLI modules?

Technical: Native code is compiled with VS2010, and thus the wrapper is compiled using that compiler in VS2012. C++ CLI Project file is edited to set the desired Framework target. Test application is written in C#.

Lotharlux
  • 75
  • 7

2 Answers2

1

So we found the bug in the end. The third party communication middleware had a bug in the 64-bit version. A pointer that was cast to a long and back again, causing a 64-bit addressing error.

Why this was first provoked when using .NET 4.5, I don't know. I seems like Server 2012 R2 with .NET 4.5 always allocates memory outside a 32-bit address scope. At least on our testbed. It always got this access violation error.

Lotharlux
  • 75
  • 7
1

Server 2012 R2 with .NET 4.5 always allocates memory outside a 32-bit address scope

Yes, this is pretty visible when you use Dumpbin.exe /headers on the EXE file. I'll post the relevant part of the output it displays:

        4.00 operating system version
        0.00 image version
        6.00 subsystem version                      // <== here
           0 Win32 version
        8000 size of image
         200 size of headers
           0 checksum
           3 subsystem (Windows CUI)
        8560 DLL characteristics
               High Entropy Virtual Addresses       // <=== here
               Dynamic base
               NX compatible
               No structured exception handler
               Terminal Server Aware

The subsystem version is the first important change in .NET 4.5 compiled executables. Version 6.00 is the version of Vista, previously that was set to 4.00. One side-effect is that the program cannot run anymore on XP and Server 2003. Version 6 is the last drastic overhaul of the Windows kernel. Big changes, the kind that made Vista so unpopular.

And the High Entropy Virtual Addresses characteristic is the important change. That's the one that caused the bug in your native code to trip. This option is related to ASLR (Address Space Layout Randomization), a counter-measure against malware. By randomizing the address space layout, it becomes difficult for malware to attack code, it can no longer depend on such code to exist at a known address.

The original version of ASLR, enabled by the Dynamic base option, randomized addresses by picking one of 256 possible address offsets. Chosen at boot time. There are much better choices in a 64-bit process, it has a much larger address space. The /HIGHENTROPYVA linker option enables it. But with the side-effect that addresses are no longer likely to be located in the lower 4 gigabyte address space. Thus tripping the casting bug.

.NET 4.5 compilers turn these on when you target 4.5 and up, 4.0 compilers did not. Not having to support XP and Server 2003 anymore (they can't run 4.5) enabled these options.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536