-1

I ask for your patience about the lack of code snippets in this question and the its vagueness, but I'm totally clueless, I don't have any clue about the location of the bug, and I cannot paste an entire application.

I have a cross platform application which opens a file through the normal C API (fopen etc), writes some data (first passing the buffer to zlib to deflate it, but I don't think that this is relevant) for a consistent time, and finally flushes and close it.

This works perfectly fine on all platform, except for the 64 bit build on Windows OS x64 with UAC turned on. Basically, on that precise setup, it seems that the file buffer is literally interlaced with what I've sent to stdout between the time I open the file and I flush it, as if any write to stdout used the same buffer of the other file.

It is important to note that this shouldn't be related to any file system virtualization (the VirtualStore mechanism), as I'm writing in %USERPROFILE%\Saved Games\. The problem is surely related to UAC because if I turn it off, the problem isn't happening. No problem in wine64 also.

Any pointer is valuable. Compiler is g++ 4.7.0 (cross compiling from linux).

Lorenzo Pistone
  • 5,028
  • 3
  • 34
  • 65
  • 2
    Can you post some sample code please? – Jonathan Potter Sep 23 '12 at 21:51
  • 1
    When I say that I have no clue on the location of the bug, and so pasting some relevant code means pasting the entire application, I'm serious. – Lorenzo Pistone Sep 23 '12 at 21:54
  • 2
    SUGGESTION: can you reproduce the problem with a small, standalone test case (for example: "`printf (some-text)/fopen (some-file-in-%userprofile%-saved-games)/fprintf (some-different-text)/fclose()`"? Q: What do you mean "cross compiling gcc 4.7.0 from Linux"? And Q: Have you tried compiling directly on Windows (e.g. using MSVC?) – paulsm4 Sep 23 '12 at 21:58
  • For UAC to make a difference (unless that's just a coincidence, which is possible) there must be an Win32 API call that is failing, so make sure you're checking the return codes (or equivalent) for every library function that uses the Win32 API. – Harry Johnston Sep 23 '12 at 22:24
  • 1
    Use process monitor from sysinternals to get closer look at what really gets written and where – marcinj Sep 24 '12 at 00:18
  • 2
    One approach that is sometimes helpful in cases like this is to make a copy of your source code and then start pulling parts out wholesale (replacing them with dummy code as necessary) until either (a) the problem goes away, in which case you look at the code you most recently removed for clues; or (b) you have a standalone test case which you can post. It can be a big job so it's kind of a last resort, but if debugging hasn't given you the clues you need it may be your only option. – Harry Johnston Sep 24 '12 at 03:04
  • Without any code no one could help you. You really need to create a simple standalone test case which does the same thing. Or at least use process monitor from SysInternals, and compare the cases with UAC on and off. It's surely a call to Win32 API that fails but the failure is not handled correctly. – Alexey Ivanov Sep 24 '12 at 06:32
  • Well, process monitor is already a good hint (how come I didn't think of that before). Will post results. – Lorenzo Pistone Sep 24 '12 at 07:41

1 Answers1

0

I have found a workaround for this problem.

First blocker was that I compiled my application with -mwindows, and that prevented my application to be attached to the parent process console. But switching to -mconsole didn't just fix it up. Even with this change everything I sent to stdout wasn't being sent to the console. I didn't state this issue in the question because it wasn't a big deal, the application has another sort of console in it.

I am certain that the whole problem is due to some bug (or some very poorly documented behaviour change) in msvcrt. Basically, when my application starts up, stdout is a valid pointer, but it is not really attached to the parent console. I have to do a freopen( "CON", "wt", stdout ); to see the output. Besides, fileno(stdout) without UAC returns 1 correctly, but with UAC on, stdout is still a valid pointer, but a call to fileno(stdout) return -1! I am wondering then where is really stdout pointing to in the second case.

It is interesting to note that when compiling with -mwindows, fileno(stdout) returns 3! I don't know if the POSIX standard requires stdout to have a file descriptor number of 1, but the choice of 3 is, at least, unusual.

Lorenzo Pistone
  • 5,028
  • 3
  • 34
  • 65
  • Compiling with -mwindows instructs the linker to build a GUI application. Windows does not provide GUI applications with standard input/output streams, so in that scenario, stdout is not an open file and the result of fileno is undefined as per http://msdn.microsoft.com/en-us/library/aa246862%28v=vs.60%29.aspx – Harry Johnston Oct 01 '12 at 03:52
  • Is the application requesting UAC elevation? That could explain the different behaviours; if UAC is enabled, UAC elevation occurs and UAC (not the console) is the parent process. On the other hand, I would have expected a new console to be created in this scenario, provided you compiled with -mconsole. – Harry Johnston Oct 01 '12 at 04:02
  • On the third hand, since Visual Studio 6 didn't build AMD64 applications, the 64-bit version of msvcrt.dll is undocumented, so there might well have been a deliberate behaviour change. Note also that the POSIX standard does not apply, because msvcrt does not attempt to provide POSIX compatibility. – Harry Johnston Oct 01 '12 at 04:08
  • @HarryJohnston it doesn't require any UAC elevation. There is yet another quirck: I guess that ´freopen´ confuses ´consolehost´, because it crashes after my application quits, even with the 32 bit build. Thanks for clarification anyway, I didn't really realize that I was in undefined land. – Lorenzo Pistone Oct 01 '12 at 09:35
  • oh, another quirk. If I don't open my application through a .bat file (which opens a console), but create a direct shortcut, then yes my application opens itself a console, but it doesn't show any output. I'm really confused. – Lorenzo Pistone Oct 01 '12 at 09:43
  • I'm guessing at this point you can reproduce the problem (the fact that stdout doesn't work even with -mconsole) with a very small, simple program, and it's a different problem than the one you originally asked about, so it might be appropriate to ask a new question about using GCC to build a 64-bit Windows console applications. (If you tag it with both gcc and windows64 you should get the attention of the right people.) – Harry Johnston Oct 01 '12 at 20:57