MSBuild error MSB4018 cannot access project.assets.json in NET 5 build

Question

MSBuild error MSB4018 cannot access project.assets.json in NET 5 build operation MSBuild error MSB4018

I am building 70 C# NET 5 projects in parallel groups and sometimes get the following error on random projects within the build

error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [c:\dev\highspeed\HsCore\hscore\HsCore.csproj] C:..\sdk\6.0.202\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(172,5): error MSB4018: System.IO.IOException: The process cannot access the file 'c:\dev\highspeed\HsCore\hscore\bin\Debug\net5.0-windows7.0\win-x64\HsCore.deps.json' because it is being used by another process. [c:\dev\highspeed\HsCore\hscore\HsCore.csproj]

The Microsoft doc says: This error is emitted when a task fails with an unhandled exception. This is generally a sign of a bug in the task. Error MSB4018 may be caused when running a task in an environment that it was not prepared for, for instance when a task has an x86 dependency but is running in a 64-bit MSBuild environment. This can manifest as a System.DllNotFoundException.

In my case, I am totally in a windows x64 environment, building and using AnyCPU libraries (and publishing to win-x64, but that doesn’t matter before the build runs). I invoke the build on xxx.sln files with the following arguments:

Exit Code 1: /t:Clean;Restore;Publish /p:Platform="Any CPU" /p:Configuration=Debug /p:TargetFramework=net5.0-windows7.0 /p:RuntimeIdentifier=win-x64 "c:\path\MySolution.sln"

The code normally builds and runs fine, except for when this kind of an error occurs. Often, when I run the build a second time, the build succeeds.

I do not understand why the MSBuild process (or one of its processes) cannot access the project.assets.json file, because MSBuild is the only one who ever accesses that file in that project. None of my tools ever reference that file; Visual Studio does not have the file or project open; No other projects in the parallel build group would ever access the projects.assets.json file anyway; so MSBuild is the only one working with the entire project tree.

The best I can think of is that maybe the Restore target in the Clean;Restore;Publish chain might be locking the file and not releasing it fast enough for the Publish(Build included) operation. But wouldn’t MSBuild be smart enough to manage that kind of thing?

Does anyone have any idea what might be going on? What other process could possibly be looking at (and locking) that file? Thank you

Answer 1

After days of debugging and investigating, I am convinced the problem is caused by using MSBuild multi-core operation (-m:) when building my solution (.sln) files.

If I remove the -m MSBuild command line argument, all the weird "file being used by another process" errors go away instantly. If I enable the -m:2 argument (or higher, or unbounded -m), the errors come back, instantly.

My .sln files that fail typically have three .csproj projects - a library DLL project, a LibraryXxxTests project, and a thin console program interface to the library. The Tests and console projects typically use a project reference to the library project.

Maybe (but I can't imagine why) the project references that point to the library project open the gate for MSBuild parallelism errors. (But MSBuild -m should be smart enough to see the project references and build the library project first before the console and Tests projects that reference the library DLL, yes?)

The other kind of project that sometimes fails has only two projects (lib and tests project).

My solution file and project files do nothing special or tricky - as far as I know, they are typical C# projects with 20 - 50 C# files. The errors occur on the main DLL file (Library.DLL) produced by the build, when some process is trying to write to that .DLL file and can't do it because of a lock. I don't know why.

But I do know that if I remove the MSBuild -m command line argument, the errors go away. Back to serial builds at the solution level for me ...

UPDATE: I wanted to believe MSBuild could figure out the build order among projects in a solution file, so I did more searching on the net. I found this information in the MSBuild documentation on how it calculates builds.

Graph option

If you specify the graph build switch (-graphBuild or -graph), the ProjectReference becomes a first-class concept used by MSBuild. MSBuild will parse all the projects and construct the build order graph, an actual dependency graph of projects, which is then traversed to determine the build order. As with targets in individual projects, MSBuild ensures that referenced projects are built after the projects they depend on.

Using the -graph option with the -m option seemed to work for solution-level builds (passing .sln files to MSBuild). The graph option tells it to calculate a build order graph among projects within each solution file. The -m -graph combination seems to let me use multiple cores to do the builds in a shorter time without MSBuild getting access errors among projects within a solution.

Keep in mind that this method requires that you use ProjectReferences in your project XML files. If you use direct assembly references to the project.DLLs stored in some other location, the -graph option cannot calculate a proper build order within the solution.

UPDATE 2:

The information about using -graph above seemed to help but did not solve all the problems. I was lucky enough to find the following information on GitHub:

I'm tempted to call this a duplicate of #9585, which is itself not the first time the problem has been found but I can't find the original.

Problems arise whenever you explicitly pass a --framework (CLI) or /p:TargetFramework= (direct MSBuild) to the build of a solution. The issue is that global properties (such as that TF setting) are inherited in most cases, but not all. Here, the ClassLibrary1 project is built once with an explicit TF (inherited from the solution build) and once with no TF (since it only has one, the ProjectReference from WebApplication1 does not pass its TF down).

The best way out of this is to only pass --framework to builds of individual projects, not solutions. Project-to-project references are careful not to get into this situation.

Removing the /p:TargetFramework=win-x64 from the msbuild command line when building .SLN files seems to be working for me so far.