VStest code coverage report in jenkins

Question

I am setting CI for .Net project using Jenkins.

I used MSTest Plugin and VStestrunner plugin to run test. Now I have .trx file and .Coverage file I am facing problem in displaying code coverage report

Please help me is you know any plugin to do this.

Answer 1

I have struggled this for a long time, finally I found we can use "CodeCoverage.exe" "ReportGenarator.exe" and "Cobertura plugin" to show perfect coverage report.

"ReportGenarator.exe" can be get from https://github.com/danielpalme/ReportGenerator/releases

• first use "CodeCoverage.exe" translate .coverage file to .xml file

    "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze -output:./TestResults/coverage.xml ./TestResults/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.coverage"

• second use ReportGenarator.exe translate vstest xml format to Cobertura xml format

    "ReportGenerator_4.4.7\net47\ReportGenerator.exe" -reports:./TestResults/coverage.xml -targetdir:./TestResults -reporttypes:cobertura

• finally install cobertura plugin use it to collect xml file, here give a pipeline useage example

    post {
        always {
            cobertura coberturaReportFile: './TestResults/Cobertura.xml'
        }
    }

• the result just like this

Answer 2

To display the coverage report you need to convert it in XML format and use MSTest Plugin to publish the report. MSTest Plugin recommends (https://wiki.jenkins-ci.org/display/JENKINS/MSTest+Plugin) to use third party application to convert in XML format and powershell (you will need to install pugin for it) to run it.

However you can convert it with PowerShell only. There is example of script:

$coverageFile = $(get-ChildItem -Path .\TestResults -Recurse -Include *coverage)[0]
$xmlCoverageFile = ".\TestResults\vstest.coveragexml"

Add-Type -path "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies\Microsoft.VisualStudio.Coverage.Analysis.dll"

[string[]] $executablePaths = @($coverageFile)
[string[]] $symbolPaths = @()

$info = [Microsoft.VisualStudio.Coverage.Analysis.CoverageInfo]::CreateFromFile($coverageFile, $executablePaths, $symbolPaths);
$data = $info.BuildDataSet()

$data.WriteXml($xmlCoverageFile)

You maybe will need to fix the path to Microsoft.VisualStudio.Coverage.Analysis.dll according to your VS version.

Answer 3

Following ghking's answer, cobertura complains that the xml is not found although it's on the disk. I have to remove './' from the path, so that cobertura is able to find the file.

   post {
       always {
           cobertura coberturaReportFile: 'TestResults/Cobertura.xml'
       }
   }

Answer 4

The complete script to do this is:

<#
.SYNOPSIS
    Script to convert code coverage report into xml format that can then be published by external tools.

.DESCRIPTION
    Covering code coverage statistics as part of quality improvement initiatives.    
#>
Param(
    [String] $InputCoveragePath =@("..\GeneratedFiles\Docs\Reports"),
    [String] $OutputCoverageFileExtension =@(".coveragexml"),
    [String] $CoverageAnalysisAssembly =@("Microsoft.VisualStudio.Coverage.Analysis.dll"),
    [String[]] $ExecutablePaths =@(""),
    [String[]] $SymbolPaths =@("")
)
    $ScriptLocation = Split-Path $script:MyInvocation.MyCommand.Path -Parent
    Write-Host $ScriptLocation

$RunAs32Bit = {
    Param(
        [String] $InputCoveragePath =@("..\GeneratedFiles\Docs\Reports"),
        [String] $OutputCoverageFileExtension =@(".coveragexml"),
        [String] $CoverageAnalysisAssembly =@("Microsoft.VisualStudio.Coverage.Analysis.dll"),
        [String[]] $ExecutablePaths =@(""),
        [String[]] $SymbolPaths =@(""),
        [String] $ScriptLocation =@(".")
    )
    Write-Host "[CoverageConverter][Begin]: Coverage conversion started..."

    Write-Host "[CoverageConverter][InputCoveragePath]: $InputCoveragePath"
    Write-Host "[CoverageConverter][OutputCoverageFileExtension]: $OutputCoverageFileExtension"
    Write-Host "[CoverageConverter][CoverageAnalysisAssembly]: $CoverageAnalysisAssembly"
    Write-Host "[CoverageConverter][ExecutablePaths]: $ExecutablePaths"
    Write-Host "[CoverageConverter][SymbolPaths]: $SymbolPaths"
    Write-Host "[CoverageConverter][ScriptLocation]: $ScriptLocation"
    
    Add-Type -path "$CoverageAnalysisAssembly"

    $Result = 0
    if($InputCoveragePath -and (Test-Path "$InputCoveragePath") )
    {
        [string[]] $coverageFiles = $(Get-ChildItem -Path $InputCoveragePath -Recurse -Include *coverage)
        
        @($coverageFiles) | ForEach-Object {
            $coverageFile = $_
            $coverageFileOut = (Join-Path -Path $(Split-Path $_ -Parent) -ChildPath  ($(Get-Item $_).BaseName + "$OutputCoverageFileExtension"))

            Write-Host "If all OK the xml will be written to: $coverageFileOut"

            $info = [Microsoft.VisualStudio.Coverage.Analysis.CoverageInfo]::CreateFromFile($coverageFile, $ExecutablePaths, $SymbolPaths);
            if($info){
                $data = $info.BuildDataSet()
                $data.WriteXml($coverageFileOut)
            }
        }
    }
    else
    {
        Write-Host "Please specify a valid input coverage file."
        $Result = 1
    }

    Write-Host "[CoverageConverter][End]: Coverage conversion completed with result $Result"
    return $Result  
}

#Run the code in 32bit mode if PowerShell isn't already running in 32bit mode
If($env:PROCESSOR_ARCHITECTURE -ne "x86"){
    Write-Warning "Non-32bit architecture detected, processing original request in separate 32bit process."
    $Job = Start-Job $RunAs32Bit -RunAs32 -ArgumentList ($InputCoveragePath, $OutputCoverageFileExtension, $CoverageAnalysisAssembly, $ExecutablePaths, $SymbolPaths, $ScriptLocation)
    $Result = $Job | Wait-Job | Receive-Job
}Else{
    $Result = Invoke-Command -ScriptBlock $RunAs32Bit -ArgumentList ($InputCoveragePath, $OutputCoverageFileExtension, $CoverageAnalysisAssembly, $ExecutablePaths, $SymbolPaths, $ScriptLocation)
}