How to trouble-shoot missing test coverage in TeamCity?

Question

According to the log I'm missing the info to get the test coverage. Possible reasons are:

1. Include / exclude patterns are incorrect
2. Assemblies are compiled without debugging information
3. PDB files are not available
4. Visual Studio code coverage is enabled for MSTest
5. TESTRUNCONFIG is used for MSTest and Visual Studio code coverage is not disabled

Being a novice on TeamCity, I need some guidance on what to do.

1. I've included all DLLs by +:*DLL. No exclusion at the moment but I've tried to omit the test itself by -:*test*.
2. I compile using DEBUG profile and the result is put on the server in bin/debug and obj/debug directories. There's no release at all.
3. I have PDB files in .source/bin/debug on the server.
4. Since I run the nUnit and dotCover on the TeamCity server, there's no VS involved.
5. Since I run the nUnit and dotCover on the TeamCity server, there's no VS involved.

What do I miss? What can I do more?

Edit

The system is: VS13, TFS10, TC8.2 (the latest available with the included nUnit and dotCoverage)

The part of log that is the testing step (no errors in the compile step reported, exit code 0). After this, there's only the removal of dotCover snapshot files. Note, however, that there's no DATA file to be found and that the XML file contains no data.

<?xml version="1.0" encoding="UTF-8"?>
<Root DotCoverVersion="2.6.1000.602" 
      ReportType="TeamCityXml" 
      CoveragePercent="0" 
      TotalStatements="0" 
      CoveredStatements="0"/>

What can I do here?

[10:59:39]Step 2/2: Test (NUnit) (29s)
[10:59:39]Starting: C:\TeamCity\buildAgent\plugins\dotnetPlugin
\bin\JetBrains.BuildServer.NUnitLauncher.exe #TeamCityImplicit
[10:59:39]in directory: C:\TeamCity\buildAgent\work\263aa919ed5f7bb8
[10:59:46]JetBrains dotCover Console Runner v2.6.1000.602. Copyright (c) 2009-2014 JetBrains s.r.o. All rights reserved.
[10:59:55][JetBrains dotCover] Coverage session started [2014-06-30 10:59:55]
[11:00:02]Start TeamCity NUnit Test Runner
[11:00:02]Running NUnit-2.6.3 tests under .NET Framework v4.0 x64
[11:00:03]AutonomousTesting.dll
[11:00:03]CoverageTest.TestExample.TestEquality
[11:00:03]CoverageTest.TestExample.TestException
[11:00:03]CoverageTest.TestExample.TestInequality
[11:00:03]CoverageTest.TestExample.TestOmission
[11:00:03]Test ignored: CoverageTest.TestExample.TestOmission
[11:00:05]AutonomousTesting.dll
[11:00:05]CoverageTest.TestExample.TestEquality
[11:00:05]CoverageTest.TestExample.TestException
[11:00:05]CoverageTest.TestExample.TestInequality
[11:00:05]CoverageTest.TestExample.TestOmission
[11:00:05]Test ignored: CoverageTest.TestExample.TestOmission
[11:00:07][JetBrains dotCover] Coverage session finished [2014-06-30 11:00:07]
[11:00:07][JetBrains dotCover] Coverage results post-processing started [2014-06-30 11:00:07]
[11:00:08][JetBrains dotCover] Coverage results post-processing finished [2014-06-30 11:00:08]
[11:00:09]##teamcity[importData type='dotNetCoverage' tool='dotcover' file='C:\TeamCity\buildAgent\temp\buildTmp\coverage_dotcover16594618384737853441.data']
[11:00:09]Importing data from 'C:\TeamCity\buildAgent\temp\buildTmp\coverage_dotcover16594618384737853441.data' (8.38 KB) with 'dotNetCoverage' processor [11:00:09]Process exited with code 0
[11:00:09]Waiting for 1 service processes to complete
[11:00:09]Processing 1 coverage report(s)
[11:00:09]Generating coverage report by dotcover for files: [C:\TeamCity\buildAgent\temp\buildTmp\coverage_dotcover16594618384737853441.data]
[11:00:09]Get dotCover version
[11:00:09]Started dotCover: C:\TeamCity\buildAgent\tools\dotCover\dotCover.exe version C:\TeamCity\buildAgent\temp\buildTmp\dotCover4472367238745438467Version
[11:00:09]Output: JetBrains dotCover Console Runner v2.6.1000.602. Copyright (c) 2009-2014 JetBrains s.r.o. All rights reserved.
[11:00:09]dotCover exited with code: 0
[11:00:09]Use DotCover 2.6.x commands set
[11:00:09]Merge dotCover reports (9s)
[11:00:19]Started dotCover: C:\TeamCity\buildAgent\tools\dotCover\dotCover.exe merge C:\TeamCity\buildAgent\temp\buildTmp\dotcover4719506578346509917.xml
[11:00:19]Output: JetBrains dotCover Console Runner v2.6.1000.602. Copyright (c) 2009-2014 JetBrains s.r.o. All rights reserved. [JetBrains dotCover] Snapshot merging started [2014-06-30 11:00:18] [JetBrains dotCover] Source snapshots number: 1 [JetBrains dotCover] Snapshot merging finished [2014-06-30 11:00:19]
[11:00:19]dotCover exited with code: 0

Answer 1

Konrad, I unfortunately do not know enough (anything?) about .net or team city to tell you what is wrong with your setup- so this is going to be more of a guideline of what I would look at if I had to fix it.

Coverage works by instrumenting compiled binaries so that they spit out information as they run. When it is set up, your unit tests will, while running, be outputting information into a results file of some sort. After the tests finish, you can analyze that results file.

The tutorial in demoncodemonkey's answer makes it seem like TeamCity is actually doing the test coverage. I am pretty sure that this isn't the case. TeamCity is most likely interfacing with your tools and hiding the nitty gritty from you. But if its just not working, having the details hidden can be less than helpful!

My first step would be to figure out how to run the tool from the command line without TeamCity. Once you can do that, you can figure out what TeamCity config you need to use to get the same result. This link is the dotCover CLI documentation.

http://www.jetbrains.com/dotcover/webhelp/dotCover__Setting_up_Coverage_Analysis_JetBrains_TeamCity.html

We use Jenkins and a lot of linux type tools- but trying to approximate the build/test by hand has always been the quickest way to figure out why it doesn't work on the server.

Answer 2

I followed this nice little tutorial to get it working for me. Here's something it mentions you could try if you're not getting any results:

Little hint: If you do everything right but no report will be generated take a look into the BuildLog. At the first try I got this error:

Solution:

Failed to read source file >'C:\TeamCity\buildAgent\temp\buildTmp\dotcover8583844779204955574.xml'. Could not find a part of the path 'C:\Windows\system32\config\systemprofile\AppData\Local\Temp\4q-kqg6z.tmp'.

Create the searched “Temp” folder in “C:\Windows\system32\config\systemprofile\AppData\Local”

Normally it doesn´t exist and because of this the error appeared. After this it works.

If that doesn't work then you'll probably need to provide more information in order for us to help you deduce what's wrong.

e.g what are your paths, what versions of everything are you installed, what settings have you got enabled for your build configurations, etc.

---

Edit: As you mentioned you already had this working, I wonder if you still have the build logs of the previously working builds? Before your colleague messed it up. Is there any chance you can do a diff between the build logs when it was working and when it's not? It might give you a clue of what he changed.

---

I managed to make a brand new configuration, to test the minimum work required to generate coverage. Here's exactly the steps I took. Maybe there's something here that you will find you've done differently.

From the main TeamCity screen at localhost:8080, click "Create a Project"

• Name: MyProjectName
• Click "Create"

Click "Add A Build Configuration"

• Name: Debug

Click "VCS Settings"

Click "Create And Attach VCS Root"

• Type of VCS: Git
• VCS Root Name: MyProjectName Git Repo
• Fetch URL: https://myusername@bitbucket.org/myusername/myprojectname.git
• Auth method: Password (enter bitbucket username and password)
• Checking Interval: custom, 600 seconds
• Click "Test Connection", it should go green. Click "Save"

Click "Add Build Step"

• Runner Type: MSBuild
• Step name: Build MyProjectName
• Build file path: MyProjectName.sln
• MSBuild Version: Microsoft .NET Framework 4.5
• MSBuild ToolsVersion: 4.0
• Run platform: x64
• Click "Save"

Click "Add Build Step"

• Runner Type: NUnit
• Step name: Run MyProjectName Tests
• .NET Runtime Platform: x64
• .NET Runtime Version: 4.0
• Run tests from: MyProjectName.Tests\bin\Debug\MyProjectName.Tests.dll
• .NET Coverage Tool: Jetbrains dotCover
• Click "Save"

Click "Projects"

Click "Run..."

After all that, the build should have generated the coverage. Good luck!

Answer 3

The problem that I had was that although my tests executed and I got results from them, there was no test coverage showing in the results. The cause for this was that in the coverage Assemblies Filters selection, I was using:

+:MySolution.*.dll
+:MySolution.*.exe
-:MySolution.*.Tests.dll

However, for coverage results to be calculated, you have to remove the .DLL extension from the filters, so that it becomes:

+:MySolution.*
-:MySolution.*.Tests

Then results came magically flowing through.