Qt Moc'ing multiple files in parallel under msbuild

Question

Using the Qt Visual studio plugin it seems to take care of most of almost everything in a seemless manner. Unfortunately it does the moc'ing using a CustomBuild step in msbuild. This results in a serial moc'ing, one after another. Is there a way to convince msbuild to do them in parallel - I'm tired of 7 cores sitting on their laurels like me.

I've seen msbuild's BuildInParallel, but I'm not sure how to make that apply here.

msbuild snippet:

<Project>
...
  <ItemGroup>
    <CustomBuild Include="a_class4.h">
      <AdditionalInputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">$(QTDIR)\bin\moc.exe;%(FullPath)</AdditionalInputs>
      <Message Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Moc%27ing %(Identity)...</Message>
      <Outputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">.\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp</Outputs>
      <Command Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">"$(QTDIR)\bin\moc.exe"  "%(FullPath)" -o ".\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp"  -DUNICODE -DWIN32 -DQT_LARGEFILE_SUPPORT -DQT_CORE_LIB -DQT_GUI_LIB "-I." "-I.\GeneratedFiles" "-I$(QTDIR)\include" "-I.\GeneratedFiles\$(ConfigurationName)\." "-I$(QTDIR)\include\qtmain" "-I$(QTDIR)\include\QtCore" "-I$(QTDIR)\include\QtGui" "-I." "-I." "-I." "-I."</Command>
    </CustomBuild>
    <CustomBuild Include="a_class3.h">
      <AdditionalInputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">$(QTDIR)\bin\moc.exe;%(FullPath)</AdditionalInputs>
      <Message Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Moc%27ing %(Identity)...</Message>
      <Outputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">.\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp</Outputs>
      <Command Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">"$(QTDIR)\bin\moc.exe"  "%(FullPath)" -o ".\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp"  -DUNICODE -DWIN32 -DQT_LARGEFILE_SUPPORT -DQT_CORE_LIB -DQT_GUI_LIB "-I." "-I.\GeneratedFiles" "-I$(QTDIR)\include" "-I.\GeneratedFiles\$(ConfigurationName)\." "-I$(QTDIR)\include\qtmain" "-I$(QTDIR)\include\QtCore" "-I$(QTDIR)\include\QtGui" "-I." "-I." "-I." "-I."</Command>
    </CustomBuild>
  </ItemGroup>
 ...
</Project>

score 0 · Answer 1 · answered Jul 18 '12 at 13:01

0

You can build in parallel using jom:
http://labs.qt.nokia.com/2009/03/27/speeding-up-visual-c-qt-builds/

answered Jul 18 '12 at 13:01

svlasov

9,923
2
38
39

Unfortunately that is only if you are using `nmake`, I don't believe it applies if you are using `msbuild`. – Zac Jul 18 '12 at 21:30
@Zac - Sorry if I missed this, but did that standalone snippet I shared capture / implement your intention ? – Parag Doke Sep 20 '15 at 04:04

score 0 · Answer 2 · answered Apr 14 '15 at 16:14

I am not very familiar with Visual Studio projects ... so don't know if the "CustomBuild" ItemGroup name and its metadata names bear significance. If they do, you might need to

Modify the ItemGroup name to something else
Identify the msbuild targets file which handles that and fix it for parallel processing

Here is a standalone msbuild file that can execute things in parallel.

Notes:

The ping -n 30 -w 1000 127.0.0.2>nul command has been added so that I could launch process explorer and find enough time to count how many instances of ping.exe were spawned.
I was trying to ping 127.0.0.2 because I wanted ping not to finish earlier.
There is nothing practically useful happening in the file ... only echoing properties as a proof of concept.

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="RunMe">
    <ItemGroup>
        <CustomBuild Include="a_class4.h">
            <AdditionalInputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">$(QTDIR)\bin\moc.exe;%(FullPath)</AdditionalInputs>
            <Message Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Moc%27ing %(Identity)...</Message>
            <Outputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">.\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp</Outputs>
            <Command Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">"$(QTDIR)\bin\moc.exe"  "%(FullPath)" -o ".\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp"  -DUNICODE -DWIN32 -DQT_LARGEFILE_SUPPORT -DQT_CORE_LIB -DQT_GUI_LIB "-I." "-I.\GeneratedFiles" "-I$(QTDIR)\include" "-I.\GeneratedFiles\$(ConfigurationName)\." "-I$(QTDIR)\include\qtmain" "-I$(QTDIR)\include\QtCore" "-I$(QTDIR)\include\QtGui" "-I." "-I." "-I." "-I."</Command>
        </CustomBuild>
        <CustomBuild Include="a_class3.h">
            <AdditionalInputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">$(QTDIR)\bin\moc.exe;%(FullPath)</AdditionalInputs>
            <Message Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Moc%27ing %(Identity)...</Message>
            <Outputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">.\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp</Outputs>
            <Command Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">"$(QTDIR)\bin\moc.exe"  "%(FullPath)" -o ".\GeneratedFiles\$(ConfigurationName)\moc_%(Filename).cpp"  -DUNICODE -DWIN32 -DQT_LARGEFILE_SUPPORT -DQT_CORE_LIB -DQT_GUI_LIB "-I." "-I.\GeneratedFiles" "-I$(QTDIR)\include" "-I.\GeneratedFiles\$(ConfigurationName)\." "-I$(QTDIR)\include\qtmain" "-I$(QTDIR)\include\QtCore" "-I$(QTDIR)\include\QtGui" "-I." "-I." "-I." "-I."</Command>
        </CustomBuild>
    </ItemGroup>
    <Target Name="RunMe">
        <Message Text="First populate an ItemGroup such that we can use it for parallel processing"/>
        <ItemGroup>
            <InputForParallelInvoke Include="$(MSBuildThisFileFullPath)">
                <Properties>InputFile=%(CustomBuild.Identity);
                        AdditionalInputs=%(CustomBuild.AdditionalInputs);
                        Message=%(CustomBuild.Message);
                        Outputs=%(CustomBuild.Outputs);
                        Command=%(CustomBuild.Command)</Properties>

            </InputForParallelInvoke>
        </ItemGroup>
        <MSBuild Projects="@(InputForParallelInvoke)" Targets="InvokeInParallel" BuildInParallel="True" />
    </Target>
    <Target Name="InvokeInParallel">
        <Exec Command="ping -n 30 -w 1000 127.0.0.2>nul &amp; echo Properties=$(InputFile) ##### $(AdditionalInputs) ##### $(Message) ##### $(Outputs) ##### $(Command)"/>
    </Target>
</Project>

When I invoke it on via msbuild thus msbuild /m \path\to\file /p:Configuration=Debug;Platform=Win32 I see 2 instances of ping.exe.

Hope this helps solve your problem.

I tried this by adding ` InputFile=%(CustomBuild.Identity); AdditionalInputs=%(CustomBuild.AdditionalInputs); Message=%(CustomBuild.Message); Outputs=%(CustomBuild.Outputs); Command=%(CustomBuild.Command) ` — Bim, Sep 18 '15 at 11:42
and ` ` to "C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.CppCommon.targets", but always get the error "error MSB4184: The expression "[System.IO.Path]::GetDirectoryName('')" cannot be evaluated". It would be sweet if ti worked! Any ideas? The reason is that I wanted parallel execution for all custom builds. — Bim, Sep 18 '15 at 11:44
@Bim - Sorry I don't follow. Note that `$(MSBuildThisFileFullPath)` will resolve to the file where you place your stuff in ... not `C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.CppCommon.targets`. Also, it is not clear if you tried the sample standalone content I shared before. — Parag Doke, Sep 20 '15 at 04:02
I did not try the standalone sample, but added the code to build the group and do the parallel compile to the `Microsoft.CppCommon.targets` file, which is included into my .vxproj file indirectly via `Microsoft.Cpp.targets`. The file has a section that does the custom build for all items in a project file added with `CustomBuild` (it starts with the line ` — Bim, Sep 21 '15 at 07:53

Qt Moc'ing multiple files in parallel under msbuild

2 Answers2

Linked