0

I am working on an Azure Pipeline running on a Windows Self Hosted agent. My goal is to run automated tests and see the results in the Azure Pipeline UI.

Two of my tasks have error messages, here is my pipeline:

stages:
  - stage: TEST
    jobs:
      - job: Test
        steps:
        - task: NuGetAuthenticate@0
          inputs:
            nuGetServiceConnections: 'NuGetSvcConnec'
            forceReinstallCredentialProvider: true

        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: |
              echo '##vso[task.setvariable variable=TASKLIB_TEST_TOOLRUNNER_EXITDELAY]90000'

        - task: DotNetCoreCLI@2
          displayName: 'Run tests'
          inputs:
            command: test
            projects: 'src/API.Test.Core/API.Test.Core.csproj'
            arguments: '--logger trx --results-directory $(Agent.BuildDirectory)\TestResults --collect "Code coverage"'
            testRunTitle: 'Tests'
            feedsToUse: 'select'
            vstsFeed: '33706***8878fd368'
            includeNuGetOrg: false
            publishTestResults: false

        - task: PublishTestResults@2
          displayName: 'Publish Test Results'
          condition: failed() 
          inputs:
            testResultsFormat: 'VSTest'
            testResultsFiles: '*.trx'
            searchFolder: '$(Agent.BuildDirectory)\TestResults'

I had found as a workaround to set the TASKLIB_TEST_TOOLRUNNER_EXITDELAY environment variable but either my script do it wrong or, it doesn't help in my case.

The DotNetCoreCLI@2 tasks fails with the "STDIO streams have closed for tool" error and there is a warning about ".NET 5...", the project does compiles with DotNet 3 that is installed on my agent, could it be the issue ?

##[debug]Exit code 1 received from tool 'C:\Program Files\dotnet\dotnet.exe'
##[debug]STDIO streams have closed for tool 'C:\Program Files\dotnet\dotnet.exe'
##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
##[debug]Processed: ##vso[task.issue type=error;]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
##[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7)...
##[debug]task result: Failed

##[error]Dotnet command failed with non-zero exit code on the following projects : C:\Agent\_work\11\s\src\API.Test.Core\API.Test.Core.csproj
##vso[task.issue type=error;]Dotnet command failed with non-zero exit code on the following projects : C:\Agent\_work\11\s\src\API.Test.Core\API.Test.Core.csproj
##[debug]Processed: ##vso[task.complete result=Failed;]Dotnet command failed with non-zero exit code on the following projects : C:\Agent\_work\11\s\src\API.Test.Core\API.Test.Core.csproj
##[section]Finishing: Run tests

The PublishTestResults@2 task ends with green but there is an error about: "STDIO streams have closed for tool" and also: "Moved Temporarily"

##[debug]C:\Agent\_work\_tasks\PublishTestResults_0b0f0***9b1\2.180.0\modules\TestResultsPublisher.exe arg: ["@C:\\Agent\\_work\\_temp\\5f4d53***2b.txt"]
##[debug]exec tool: C:\Agent\_work\_tasks\PublishTestResults_0b0***9b1\2.180.0\modules\TestResultsPublisher.exe
##[debug]arguments:
##[debug]   @C:\Agent\_work\_temp\5f4d***bf2b.txt
[command]C:\Agent\_work\_tasks\PublishTestResults_0b0f***f9b1\2.180.0\modules\TestResultsPublisher.exe @C:\Agent\_work\_temp\5f4d***bf2b.txt
##[debug]Exception while fetching feature flag value
##[debug]One or more errors occurred.
##[debug]Exit code 20000 received from tool 'C:\Agent\_work\_tasks\PublishTestResults_0b0f0***f9b1\2.180.0\modules\TestResultsPublisher.exe'
##[debug]STDIO streams have closed for tool 

'C:\Agent\_work\_tasks\PublishTestResults_0***9b1\2.180.0\modules\TestResultsPublisher.exe'
##[debug]Exit code of TestResultsPublisher: 20000
##[debug]Failed to get FF TestManagement.Server.UsePublishTestResultsLibInAgent Value.
Unable to get the FF: TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask. Reason: One or more errors occurred. (Moved Temporarily)

##[debug]Failed to get FF TestManagement.PTR.CalculateTestRunSummary Value.
##[debug]Reading test results from file 'C:\Agent\_work\11\TestResults\***_2021-04-12_14_19_44.trx'
##[debug]Setting run start and finish times.
##[debug]Attachment location: C:\Agent\_work\11\TestResults\***_2021-04-12_14_19_44\In
##[debug]Adding run level attachment: C:\Agent\_work\11\TestResults\***_2021-04-12_14_19_44\In\***\***-1_2021-04-12.14_19_37.coverage
##[debug]Processed: ##vso[results.publish type=VSTest;mergeResults=false;publishRunAttachments=true;resultFiles=E:\Agent\_work\11\TestResults\AzureBuild_Svc_DEVOPS-DLP-1_2021-04-12_14_19_44.trx;failTaskOnFailedTests=false;testRunSystem=VSTS - PTR;]
##[debug]task result: Succeeded
##[debug]Processed: ##vso[task.complete result=Succeeded;]
##[debug]Release.ReleaseUri=undefined
##[debug]Release.ReleaseId=undefined
##[debug]Build.BuildUri=vstfs:///Build/Build/9933
##[debug]Build.Buildid=9933
##[warning]Failed to publish test results: Moved Temporarily
##[debug]Processed: ##vso[telemetry.publish area=TestExecution;feature=PublishTestResultsTask]{"builduri":"vstfs:///Build/Build/9933","buildid":"9933","osType":"Windows_NT","testRunner":"VSTest","failTaskOnFailedTests":"false","mergeResultsUserPreference":"false","config":null,"platform":null,"testResultsFilesCount":1,"dotnetVersion":"5.0.100","subFeature":"publishTestResultsTaskConsolidatedCiEvent"}
##[section]Async Command Start: Publish test results
##[section]Async Command End: Publish test results

############## Update 1 #########################

I ran locally:

"dotnet.exe" test "***\s\src\API.Test.Core\API.Test.Core.csproj" --logger trx --results-directory "***\TestResults" --collect "Code coverage"

I got the same results as from the Pipeline:

Results File: ***\TestResults\***_2021-04-13_14_05_46.trx

Attachments:
  ***\TestResults\d91***7774c\***_2021-04-13.14_05_42.coverage
Failed!  - Failed:    42, Passed:     0, Skipped:     0, Total:    42, Duration: 2 s - API.Test.Core.dll (netcoreapp3.1)

The command stops there where in the pipeline it continues with:

##[debug]Exit code 1 received from tool 'C:\Program Files\dotnet\dotnet.exe'
##[debug]STDIO streams have closed for tool 'C:\Program Files\dotnet\dotnet.exe'
##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1

I don't know what action I could do locally to replicate this action, could you help me ?

From the start, is seems like it has a hard time sending back to Azure DevOps the tests and coverage results, any idea what could cause this ? I run behind a proxy but I did provide credentials and proxy values...

I'll see about reinstalling the agent... thanks.

############## Update 2 #########################

I updated my pipeline:

  - job: Test
    steps:
    - task: NuGetAuthenticate@0
      inputs:
        nuGetServiceConnections: 'MyServiceConnection'
        forceReinstallCredentialProvider: true

    - task: PowerShell@2
      displayName: 'Set Env variable to prevent error: ##[debug]STDIO streams have closed for tool C:\Program Files\dotnet\dotnet.exe'
      inputs:
        targetType: 'inline'
        script: |
          echo '##vso[task.setvariable variable=TASKLIB_TEST_TOOLRUNNER_EXITDELAY]90000'

    - task: DotNetCoreCLI@2
      displayName: 'Restore'
      inputs:
        command: restore
        projects: 'src/API.Test.Core/API.Test.Core.csproj'
        feedsToUse: 'select'
        vstsFeed: '33706e***68'
        includeNuGetOrg: false

    - task: DotNetCoreCLI@2
      displayName: 'Run tests'
      inputs:
        command: test
        projects: 'src/API.Test.Core/API.Test.Core.csproj'
        arguments: '--collect "Code coverage"'
        includeNuGetOrg: false
        publishTestResults: true

and I got this log:

##[debug]adjustedPattern: 'C:\Agent\_work\_temp\**/*.trx'
##[debug]1 final results

##[debug]Failed to get FF TestManagement.Server.UsePublishTestResultsLibInAgent Value.
Unable to get the FF: TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask. Reason: One or more errors occurred. (Moved Temporarily)

##[debug]Failed to get FF TestManagement.PTR.CalculateTestRunSummary Value.

##[debug]Reading test results from file 'C:\Agent\_work\_temp\***-04-15_18_56_07.trx'
##[debug]Setting run start and finish times.
##[debug]Attachment location: C:\Agent\_work\_temp\***-04-15_18_56_07\In
##[debug]Adding run level attachment: C:\Agent\_work\_temp\***-04-15_18_56_07\In\***\***-04-15.18_56_00.coverage
##[debug]Total test results: 42

##[debug]Processed: ##vso[results.publish type=VSTest;mergeResults=false;publishRunAttachments=true;resultFiles=C:\Agent\_work\_temp\***-04-15_18_56_07.trx;testRunSystem=VSTS - dotnet;]
##[debug]task result: Failed

##[error]Dotnet command failed with non-zero exit code on the following projects : C:\Agent\_work\11\s\src\API.Test.Core\API.Test.Core.csproj
##[debug]Processed: ##vso[task.issue type=error;]Dotnet command failed with non-zero exit code on the following projects : C:\Agent\_work\11\s\src\API.Test.Core\API.Test.Core.csproj
##[debug]Processed: ##vso[task.complete result=Failed;]Dotnet command failed with non-zero exit code on the following projects : C:\Agent\_work\11\s\src\API.Test.Core\API.Test.Core.csproj
Async Command Start: Publish test results
##[warning]Failed to publish test results: Moved Temporarily
Async Command End: Publish test results

I also tried from the DotNet command line, adding this option: "--logger trx --results-directory "C:\Agent_work_temp"" but can this upload the reports back to the pipeline ? Seems not possible to me... Many thanks for your help.

ClaudeVernier
  • 427
  • 4
  • 20

1 Answers1

1

In order to narrow down the issue, you could try the following items:

  1. The test command in DotNetCoreCLI does not recognize the feedRestore or vstsFeed arguments and feeds specified in this manner will not be included in the generated NuGet.config file when the implicit restore step runs. It is recommended that an explicit dotnet restore step be used to restore packages. The restore command respects the feedRestore and vstsFeed arguments.

  2. Since you are using a Windows Self Hosted agent, please try to login to the agent machine, and run .NetCore CLI manually, to see how's the result.

  3. Check the agent version and upgrade to the latest version, then try to restart the build agent service to see how's the result.

  4. Try to configure a new agent to see how's the result.

Cece Dong - MSFT
  • 29,631
  • 1
  • 24
  • 39
  • Thanks. I did as mentionned in 1 and 2. I'll explain more in Update 1 – ClaudeVernier Apr 13 '21 at 12:16
  • I installed another agent on that server, side by side with the first one, new agent version: '2.184.2') and the error is the same :( – ClaudeVernier Apr 13 '21 at 14:30
  • It seems there is not issue when run .NetCore CLI manually. Could you try to configure an agent on another machine? – Cece Dong - MSFT Apr 14 '21 at 09:00
  • It took already very long to setup this machine with security team so it can access Azure Devops and GitHub... If I could find more information on how it tries to upload the test and coverage reports back to Azure DevOps, maybe I could find what blocks it... What do you think ? – ClaudeVernier Apr 14 '21 at 12:18
  • You could try to set `publishTestResults: true` in " DotNetCoreCLI" and remove "PublishTestResults" task to see how's the result. With setting `publishTestResults: true`, you don't need to add arguments. – Cece Dong - MSFT Apr 15 '21 at 09:46
  • I had tried that before but had errors, please see my latest tests in Update 2 – ClaudeVernier Apr 15 '21 at 17:07
  • Do you use Azure DevOps service (https://dev.azure.com/xxxx) or on-premises Azure DevOps server? – Cece Dong - MSFT Apr 16 '21 at 09:03
  • Hello, we use DevOps service. – ClaudeVernier Apr 18 '21 at 16:04
  • It seems your issue is on "publish test result", you are not be able to publish test result to DevOps pipeline. Could you see other pipeline have this issue? – Cece Dong - MSFT Apr 19 '21 at 11:08
  • This is my first pipeline with automated tests, I can try building a test project to see if different. Does it require special openings in firewall ? – ClaudeVernier Apr 19 '21 at 15:50
  • You may try to create a `.proxybypass` file in the agent's root directory as this documentation describe, and make sure the agent can initiate communication with the URLs and IP addresses describe in this link: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops#im-running-a-firewall-and-my-code-is-in-azure-repos-what-urls-does-the-agent-need-to-communicate-with. Last, please try to run your project on Microsoft-Hosted agent and check whether it works or not. – Cece Dong - MSFT Apr 20 '21 at 09:50
  • How's your issue going? – Cece Dong - MSFT Apr 27 '21 at 09:51
  • Hello, thanks for asking ! I admit I was doing a break on this... thanks for the link, I'll submit the list to our security team so they open-up the doors for me ! :) – ClaudeVernier Apr 27 '21 at 15:02
  • I understand your comment and I thank you but for now, I still have both error message... STDIO streams have closed for tool && Moved Temporarily – ClaudeVernier Apr 28 '21 at 06:34
  • That's fine, if you have any update, kindly let us know. – Cece Dong - MSFT Apr 28 '21 at 06:35