In this case I do have the source for both the application and the dll.
When both are compiled without /NXCOMPAT, they work together fine. But when I compile both with /NXCOMPAT, I get a segfault deep in kernelspace.
If I compile the dll with /NXCOMPAT, and compile the executable without, it also works fine. (not surprising I suppose, since the DEP settings for the executable get forced on the loaded dll.)
I have previously seen a segfault in MainCRTStartup (Note: not the dll version), after enabling DEP, which was caused by another linker option. However, in this case that other linker option is NOT set, so I know that's not the answer.
Anyone have an idea where I should look for the cause?
Edit: Further strangeness. I've been running this in the debugger in VS 2008 the whole time, but when I tried running it without the debugger attached, the segfault disappears. I find this a very unsatisfactory solution, as I still don't know why it's been doing this.
Edit the 2nd: Also segfaults running in the debugger in VS 2013 Express.