5

We use dotnet test --no-build to run our unit tests during PR/CI builds:

- task: DotNetCoreCLI@2
  ${{ if eq(parameters.shortName, '') }}:
    displayName: "Test ${{ parameters.name }}"
  ${{ if ne(parameters.shortName, '') }}:
    displayName: "Test ${{ parameters.shortName }}"
  inputs:
    command: "test"
    projects: ${{ parameters.path }}
    ${{ if eq(parameters.testFilter, '') }}:
      arguments: $(TestBinLogArg) -c $(BuildConfiguration) --no-build ${{ parameters.commonFlags }} ${{ parameters.moreTestArgs }} --filter ${{ parameters.unitTestFilter }}
    ${{ if ne(parameters.testFilter, '') }}:
      arguments: $(TestBinLogArg) -c $(BuildConfiguration) --no-build ${{ parameters.commonFlags }} ${{ parameters.moreTestArgs }} --filter (${{ parameters.unitTestFilter }})&(${{ parameters.testFilter }})
    publishTestResults: ${{ parameters.publishTestResults }}

As you can see we use a standard Azure DevOps task DotNetCoreCLI@2 to run our unit tests. A typical command line would be:

"C:\Program Files\dotnet\dotnet.exe" test D:\_wf\01\3\s\DataSvc.sln --logger trx --results-directory D:\_wf\01\_temp -c Debug --no-build /p:OutDir=D:\_wf\01\3\b --filter TestCategory!=NoPR|TestCategory=L0

The problem is that unit test failure messages are not surfaced as build errors: enter image description here

Yes, we can see the test failures in the Tests tab of the page or in the build logs, but I want to see them in the build summary page itself, just like we can with compilation errors when using the MSBuild@1 task (but again not DotNetCoreCLI@2 build)

We use NUnit for our tests. Any ideas?

mark
  • 59,016
  • 79
  • 296
  • 580

1 Answers1

0

dotnet test is currently not very tightly integrated with MSBuild.

The team members discuss this in this issue: https://github.com/microsoft/vstest/issues/2953#issuecomment-868368159

nohwnd may be able to explain better.

Kirill Osenkov
  • 8,786
  • 2
  • 33
  • 37
  • Thanks, I commented on the github thread. Basically I am asking to implement Exec's `CustomErrorRegularExpression` and `CustomWarningRegularExpression` as msbuild build properties that we could pass to msbuild through the `dotnet` command line. – mark Jun 27 '21 at 02:17