1

I have a solution that contains both C++ and C# projects that is built in a nightly CI build on a remote machine. The build script checks out a clean copy of the source and builds both debug and release configurations of the solution using MSBuild and runs the test suite on each configuration.

About every other build, the release configuration fails to build properly. An analysis of the build log reveals that C++ Project Q, which depends on C++ Project D, tries to link before Project D is done. This error only happens for the release configuration on this particular build machine - the debug configuration builds without error. I have a separate nightly build process that runs on a separate machine where the release configuration is built with a similar script that uses MSBuild (it just does not run the test suite), and it builds the same source revision without issue. Multiple team members build the solution without issue either from update or clean checkout with one or both configurations, always from the Visual Studio 2019 IDE, on various operating systems.

Project Q is configured with Project D as a project reference and Project D is also listed as a hard-dependency for Project Q. As I mentioned, the build script is using MSBuild.

An additional item of interest from analysis of the release and debug build logs: The build of Project D is initiated differently between the two configuration builds. It is started by its own metaproject in the release configuration (as item 60, for instance), but started earlier (as item 44, for instance), by a different project in the debug configuration. Not sure why the dependency algorithm would run with such different results in the two cases since the solution and working source being built are the same.

Any ideas or suggestions would be appreciated.

Update: Inspection of differences between the release and debug build logs reveals some interesting facts. In the failure case I did a search for ") is building" in each log - should be an indication of how many projects were built, including metaproj entries. For the failure case, debug had 282 occurrences, whereas release had 175. For the success case, debug had 280 occurrences and release had a whopping 559! A similar search for "Done Building Project" yields similar results, only off by 1 or 2. That might partly explain the differences in build order between the solutions. I also need to check for conditional build entries.

GTAE86
  • 1,780
  • 3
  • 29
  • 39
  • Did you use Project dependency(Right-click on project-->Build Dependencies) or project reference(right-click on Reference-->Add Reference) to reference project? Also, did you do any changes to your current project? If you use MSBuild to build project, you should use project reference, and also make sure that you did not have any condition to distinguish between debug and release mode. – Mr Qian May 12 '20 at 07:59
  • We use project references as needed. In some cases, that is not sufficient (like the build process for project X is dependent on a tool, project M - then we will add a dependency on X for M). In this particular case, the offending project Q has project D as both a project reference and an explicit dependency. Additionally, the solution is proven to be set up correctly because the identical source builds for the same configuration on a separate machine using the same script. Not to mention the debug configuration builds on the same machine just before the release configuration fails. – GTAE86 May 12 '20 at 16:17
  • In addition to what @MrQian pointed out, this can happen if your `AssemblyInfo.cs` file contains any `[DependencyAttribute(...)]` annotations. Any presence of these will cause an arbitrary version (i.e, "Debug" or "Release") of the assembly it mentions to be sucked into the `ResolveAssemblyReferences` phase of the dependent build, and its full path gets burned into the `*.AssemblyReference.cache` and `DesignTimeResolveAssemblyReferencesInput.cache` files. Perversely, the clean build that generates those cache files will work, but ruin and havoc lie in wait. Look inside those 2 files to see. – Glenn Slayden Jun 04 '22 at 11:48

1 Answers1

0

Visual Studio 2019 Reference Project Not Built Before Dependent Project Tries To Link in Release Configuration using MSBuild

It seems that project buid order was broken and Project D built later than Project Q which need the output content of Project D, so the whole build failed.

Not sure that if you use Project Dependency(Right-click on project-->Build Dependencies-->Project Dependencies), if so, only VS IDE Build Will recognize their build order while MSBuild command line will lose the relationship about them.

Besides, I wonder if your main project under Release mode references a dependency project under Release mode. If you did these, the build will definitely go wrong.

All of these above is that I think you made some changes to your project.

You could follow these steps:

1) I suggest you could try to use Project Reference and it will add these xml node in ProjectQ.csproj file to strongly specify build relationship:

<ItemGroup>
    <ProjectReference Include="..\ProjectD\ProjectD.csproj">
      <Project>{26c26cdd-a5e0-40c7-b0c9-4563f969424f}</Project>
      <Name>ProjectD</Name>
    </ProjectReference>
 </ItemGroup>

Also, check if there are any conditions that distinguish between Debug or Release mode when referring to a project like this:

<ProjectReference Include="..\ProjectD\ProjectD.csproj" Condition="'$(Configuration)'=='Debug'">

If so, please remove that condition Condition="'$(Configuration)'=='Debug'" to make sure it is the same between Debug and Release mode.

2) close VS Instance, delete .vs hidden folder under solution folder.

3) check your CI Build and cloud build server and ensure that cloud parameters are consistent with other servers. And examine any of your xxx.csproj files to check if you have any other operations which causes this bahavior.

In addition, if necessary, you can share your xxx.csproj file and the build script with us to troubleshoot it.

Mr Qian
  • 21,064
  • 1
  • 31
  • 41
  • Thanks for the comment. 1) - see my comment above - using project references, etc. Additionally, a checkout of the same revision to a different machine builds correctly for the release configuration using the same script. Item 2) - these are clean checkouts. The working directory is deleted as the first step. Item 3) - No cloud in the loop that I know of. Also, solution is a mixture of C++ and C# projects; problem is in C++. One thing I did yesterday on the offending build machine is upgrade from VS 2019 16.4.5 to 16.5.4. Of course it built correctly last night, but that might be unrelated. – GTAE86 May 12 '20 at 16:27
  • I will check for conditions between debug and release configurations. The configuration manager seems to be identical for both. I also added some more detail to the question. – GTAE86 May 12 '20 at 16:30
  • In reference to "All of these above is that I think you made some changes to your project." - not sure I follow. What kind of change, and when? Script deletes the working directory then checks out the source. Builds debug configuration of the solution using MSBuild, runs unit tests. It then builds release configuration (same solution) using MSBuild, runs unit tests. If you mean a possible difference between release/debug, maybe due to some conditionals in the solution, could be, but not intentionally. I brought this solution from VS 2003 to VS 2019 over 19 yrs - pretty familiar with it :) – GTAE86 May 12 '20 at 16:51
  • Actually, there is indeed an accidental build of msbuild, that is, a disorderly build. This issue is indeed rare but exists, and it is indeed a known problem. So if you want to do this step, I suggest you try to specify the order for each item, please refer to the way of [this document](https://github.com/microsoft/msbuild/issues/1957)----add such node in `xxx.sln` file. – Mr Qian May 14 '20 at 10:19