2

So response files are being used by default to specify the command line switches to the compiler and linker for some toolchains. This appears to apply only to makefiles, though. Arguably this is done to get around the 8192 character length limit for command lines on Windows, given my scenario with the NMake Makefiles generator.

Apparently the settings are supposed to be governed by CMAKE_<LANG>_USE_RESPONSE_FILE_FOR_OBJECTS which isn't exactly documented as far as I could find it. Anyway, various CMake scripts that come with the CMake (3.15.1) distribution contain either 0 or 1 for these settings (i.e. per language <LANG>). The only place where this appears to be read is cmMakefileTargetGenerator::CheckUseResponseFileForObjects() in cmMakefileTargetGenerator.cxx (as of this writing).

Complementary settings exist in the form of CMAKE_<LANG>_USE_RESPONSE_FILE_FOR_INCLUDES and CMAKE_<LANG>_USE_RESPONSE_FILE_FOR_LIBRARIES.

Now while this technically works it really causes one undesired side effect for me. In the logs from automated builds I'd like to see what was happening (yep, I am also setting CMAKE_VERBOSE_MAKEFILE=ON). Given the fact that these response files are being generated on the fly, the command lines I get to see from the log file look akin to:

C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1422~1.279\bin\Hostx86\x86\cl.exe @C:\Users\XA1DB~1.RLM\AppData\Local\Temp\nm2A99.tmp

This means I no longer get to see what arguments are being passed. The output essentially becomes becomes useless for my purposes.

How can I get CMake to generate makefiles that will also show me the contents of the response files in-line, so that I may glean the compiler/linker command line arguments from a build log?

NB: don't get me wrong, I fully appreciate the existing limitation on Windows. However, even if the actual call to invoke the compiler or linker uses response files, I'd simply like to see the response file contents instead of the response file names. I am looking for a canonical way of doing this, without having to conjure up some hacky solution that will break the next time CMake gets an update to its internals.

I also found it a bit worrying to see 8.3 (alternative) path names names used in 2019 (given fsutil behavior set disable8dot3 1 / NtfsDisable8dot3NameCreation), but perhaps the CMake authors have insights I don't have. I was unable to find any setting to configure that behavior in the most recent source code from the Git repo.

0xC0000022L
  • 20,597
  • 9
  • 86
  • 152

2 Answers2

2

nmake has an option to display inline files / response files:

/U Dump inline files

For recursive make you might want to set this in an environment variable like this:

set MAKEFLAGS=U

This way, the content of all response files will appear on the console.

jobor
  • 339
  • 1
  • 7
  • In keeping with my tradition of awarding any useful answers provided by others in Q&As that I have answered myself, I'm accepting your answer. – 0xC0000022L May 05 '20 at 21:18
0

NB: TL;DR toward the bottom.

I have meanwhile made some progress, but unfortunately the variables that - at first glance - are supposed to influence said behavior don't do a thing (or not the way one would expect from the comments), even when forcibly caching empty values for them. Excerpt from Modules\Platform\Windows.cmake:

# for nmake make long command lines are redirected to a file
# with the following syntax, see Windows-bcc32.cmake for use
if(CMAKE_GENERATOR MATCHES "NMake")
  set(CMAKE_START_TEMP_FILE "@<<\n")
  set(CMAKE_END_TEMP_FILE "\n<<")
endif()

include(Platform/WindowsPaths)

# uncomment these out to debug nmake and borland makefiles
#set(CMAKE_START_TEMP_FILE "")
#set(CMAKE_END_TEMP_FILE "")
#set(CMAKE_VERBOSE_MAKEFILE 1)

I could not find the Windows-bcc32.cmake referenced in the comment, but it stands to reason that this refers to the older name of the Borland C compiler executable. So Windows-Borland-C.cmake and Windows-Borland-CXX.cmake are the likely successors, who in turn include Windows-Embarcadero-C.cmake and Windows-Embarcadero-CXX.cmake respectively. But once I chased that clue, I found that likely the second reference to Windows-bcc32.cmake (there is another that actually references "Borland" in the comment) seems to be irrelevant.

However, what's relevant are the two variables CMAKE_START_TEMP_FILE and MAKE_END_TEMP_FILE. Setting them (even forcibly and writing to the cache) after invoking project() doesn't yield the desired effect, though.

These variables are defined in Windows.cmake (excerpt above) and used in:

  • Modules\Platform\Windows-df.cmake
  • Modules\Platform\Windows-Embarcadero.cmake
  • Modules\Platform\Windows-MSVC.cmake
  • Modules\Platform\Windows-NVIDIA-CUDA.cmake
  • Modules\Platform\Windows-OpenWatcom.cmake
  • Modules\Platform\Windows-PGI.cmake
  • Tests\RunCMake\add_link_options\LINKER_expansion-list.cmake
  • Tests\RunCMake\Make\VerboseBuild.cmake
  • Tests\RunCMake\target_link_options\LINKER_expansion.cmake
  • Source\cmMakefileTargetGenerator.cxx

It appears that CMAKE_START_TEMP_FILE and CMAKE_END_TEMP_FILE have been expanded in the places they are used, by the time project() is finished. Which means that changing Windows.cmake would be the only (ugly) viable option ... except ...

TL;DR

Add the following after your invocation of project() but be aware that if/whenever you are dealing with a large number of files/arguments on the command line, this may well hit the "ceiling" in regards to the command line length limits on Windows. But this is how you can make the commands executed by NMake visible again.

set(CMAKE_CXX_USE_RESPONSE_FILE_FOR_INCLUDES 0 FORCE)
foreach(lang IN ITEMS C CXX)
    foreach(cmd IN ITEMS COMPILE_OBJECT CREATE_SHARED_LIBRARY CREATE_PREPROCESSED_SOURCE CREATE_ASSEMBLY_SOURCE LINK_EXECUTABLE)
        string(REPLACE "${CMAKE_START_TEMP_FILE}" "" CMAKE_${lang}_${cmd} "${CMAKE_${lang}_${cmd}}")
        string(REPLACE "${CMAKE_END_TEMP_FILE}" "" CMAKE_${lang}_${cmd} "${CMAKE_${lang}_${cmd}}")
    endforeach()
endforeach()

Btw: The replacement syntax was gleaned from Tests\RunCMake\Make\VerboseBuild.cmake, by the way. I just made it into a loop to be more concise as I needed to "treat" quite a few variables this way.

What this does is to replace instances of CMAKE_START_TEMP_FILE/CMAKE_END_TEMP_FILE inside the various commands used in the NMake files with an empty string, thereby disabling the special NMake syntax for inline files. Because, as it turns out, the culprit in regards to not being able to see the commands, wasn't CMake but NMake.

Reference: Inline Files in a Makefile

Now this opens up a variety of other alternatives one could try instead. Ideas:

  • Append KEEP to CMAKE_END_TEMP_FILE in order to keep the response file around ...
  • Append a file name to use to CMAKE_START_TEMP_FILE (see "reusing inline files" and this answer).
0xC0000022L
  • 20,597
  • 9
  • 86
  • 152