[Edited: 10-Aug]
I have a project that generates a DLL (for a private NuGet package). This project is written to compile in the following frameworks:
<TargetFrameworks>netstandard2.0;netstandard2.1</TargetFrameworks>
The solution also contains four Test projects, each written in .NetCore 3.1 (consuming the .Netstandard 2.1 dll from the above project).
These has been working fine for months, but of course, I'm not testing that my NuGet package will work for code consuming the .NetStandard 2.0 framework.
I therefore updated the Test projects to compile in the following frameworks:
<TargetFrameworks>net472;net48;netcoreapp3.1</TargetFrameworks>
I immediately tripled my number of tests, and in Visual Studio they all pass (phew!). Well, almost tripled...each project contained a few tests that weren't applicable to the .NetFramework, so I used conditional compilation to remove these tests for the .NetFramework code.
All well and good....
However, when I push this up to Azure, then the test step fails.
The build script (yaml) contains the following commands:
...
- task: VSBuild@1
displayName: 'Build all'
inputs:
solution: '$(solution)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
maximumCpuCount: true
- task: DotNetCoreCLI@2
displayName: 'Run Tests'
inputs:
command: 'test'
projects: '**/*Test.csproj'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'
...
(fyi, we're also hooked into SonarCloud and report on code coverage)
The "headline" news seems promising:
Job preparation parameters
6 queue time variables used
100% tests passed
And when I dig into this I see the following pattern per test project:
Total tests: 3650
Passed: 3650
Total tests: 3650
Passed: 3650
Total tests: 3652
Passed: 3652
So, from the above, I can see from these numbers correspond to one of my test projects being executed three times, once for .Net 4.7.2, once for .Net 4.8 and once for .Core 3.1.
And I see a similar group of three executions for each of the other Test projects.
So...every test passed.
However, after the .NetCore version of each Test project executes (so, the one that had been executing fine for months), I now get the following:
##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
"C:\Program Files\dotnet\dotnet.exe" test d:\a\1\s\My.Test.Project\EMy.Test.Project.csproj --logger trx --results-directory d:\a\_temp --configuration Release --collect "Code coverage"
I don't know how to interpret it, and thus how to fix it.
My guess is that the code analysis report is receiving the coverage data three times, which is causing it some problems. If that were the case then I'd have to say in the YAML "Execute each test project once using .NetCore 3.1 as we've been doing and collect the code coverage, but then execute them all again, once for .Net 4.7.2 and then 4.8 without the code coverage". But I'm not sure how to word that in yaml...
Edit 10th Aug #1
In response to Kevin Lu's comment below, I dug this out of the details.
Data collector 'Code Coverage' message: Failed to initialize code coverage datacollector with error: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.Initialize(XmlElement configurationElement, IDataCollectionSink dataSink, IDataCollectionLogger logger) at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.OnInitialize(XmlElement configurationElement). Data collector 'Code Coverage' message: Data collector 'Code Coverage' threw an exception during type loading, construction, or initialization: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.Initialize(XmlElement configurationElement, IDataCollectionSink dataSink, IDataCollectionLogger logger) at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.OnInitialize(XmlElement configurationElement) at Microsoft.VisualStudio.TraceCollector.BaseDataCollector.Initialize(XmlElement configurationElement, IDataCollectionEvents events, IDataCollectionSink dataSink, IDataCollectionLogger logger, IDataCollectionAgentContext agentContext) at Microsoft.VisualStudio.TraceCollector.BaseDataCollector.Initialize(XmlElement configurationElement, DataCollectionEvents events, DataCollectionSink dataSink, DataCollectionLogger logger, DataCollectionEnvironmentContext environmentContext) at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectorInformation.InitializeDataCollector() at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionManager.LoadAndInitialize(DataCollectorSettings dataCollectorSettings, String settingsXml)..
Edit 10th Aug #2
I tried adding different NuGet packages to see if I could get this to work i.e. Microsoft.TestPlatform.ObjectModel and then Microsoft.TestPlatform), but to no avail.
I then changed the build script from:
- task: DotNetCoreCLI@2
displayName: 'Run Tests'
inputs:
command: 'test'
projects: '**/*Test.csproj'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'
to
- task: VSTest@2
displayName: 'Run Tests'
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\No1Test.dll
**\No2Test.dll
**\No3Test.dll
**\No4Test.dll
searchFolder: '$(System.DefaultWorkingDirectory)'
codeCoverageEnabled: true
runInParallel: true
This felt like a retrograde step, however, all tests passed and no error messages so that's something to be pleased about. But....is this the "perfect" solution?
