23

According to this schema VS2010 Premium and Ultimate has a tool for checking Code Coverage - together with a few other testing tools. Does this support nUnit too, or just MS test?

stiank81
  • 25,418
  • 43
  • 131
  • 202

9 Answers9

26

Visual Studio 2012 added support for third party unit test frameworks.

The test framework plugins available include:

  • NUnit
  • xUnit.net
  • MbUnit
  • QUnit
  • Jasmine

Adding support is very easy because you can add it from the Visual Studio Extension Manager.

For example you can add the "NUnit Test Adapter".

And now the "Analyze Code Coverage" works with third party unit framework. You can inspect all tests or only one.

And if your using VS 2010 don't be afraid to upgrade to VS2012 because sln files are compatibles (you need VS2010 with SP1).

Be.St.
  • 4,101
  • 3
  • 25
  • 35
  • 1
    This is the way to do it. I would not waste your time doing the ticked (more complex solution) when this will work fine. – Tod Thomson Apr 29 '13 at 01:15
  • 2
    Only applies to VS2012 Ultimate or Premium. `Analyze Code Coverage` is not avaliable in Professional. [source](http://msdn.microsoft.com/en-us/library/dd537628.aspx) and [source](http://social.msdn.microsoft.com/Forums/vstudio/en-US/afd3ee87-76e4-499d-ad8a-dbc6adb96cb9/visual-studio-professional-2012-theres-no-button-analyze-code-coverage) – Forty-Two Aug 16 '13 at 15:37
19

It's doable, but requires a bit of setup. I just got it working with xUnit. Presumably the below will work with NUnit too.

For this to work, you'll need three projects

  • The System Under Test -- SUT
  • A testing project using your favorite unit testing framework -- xUnitTest

  • A VS Test Project -- VSTest

    1. Create the VSTest project (breath easy)
    2. Delete the default UnitTest1.cs file
    3. Add a "Generic Test" to VSTest
    4. Specify the full path to the win32 version of the console runner for your testing framework, such as xunit.console.x86.exe
    5. Specify the full path to the xUnitTest dll
    6. Under the VS Test Menu->Edit Test Settings->Local->
    7. Data & Diagnostics "Tab"
    8. Make sure only Code Coverage is enabled
    9. Select Code Coverage row, then click the "Configure" button above (yes, this is a well designed UI :)
    10. In the "Code Coverage Detail" dialog
    11. Select both the dlls for SUT and xUnitTest
    12. Enable "Instrument assemblies in test"

Now when you run the VS2010 test, it'll correctly instrument the test dlls, run the code runner and gather the info into Visual Studio.

Scott Weinstein
  • 18,890
  • 14
  • 78
  • 115
  • 1
    Thanks! Not the smooth answer I was hoping for though.. :-P – stiank81 May 31 '10 at 15:33
  • Have been trying for several hours now. I do get the instrumented assemblies passed to nunit-console but never get code coverage results back. – LeffeBrune Jul 15 '11 at 18:27
  • 1
    There is a small detail that escaped from me -- you absolutely have to use x86 runner for your tests (such as nunit-console-x86.exe). Changing the test runner to x86 fixed issues I have been having. – LeffeBrune Jul 15 '11 at 18:55
  • I just tried this for NUnit and worked around an issue. The coverage only reported on the NUnitTest.dll and did not show up for the SUT.In step 11, instead of just selecting the SUT which is in the SUT directory I had to click add assembly and choose the SUT.exe that is in the bin directory of the NUNITTest directory. – Bentley Davis Jan 05 '12 at 17:47
5

After fiddling with this for a little over an hour, I could not get it to work properly. I was able to get the generic test running properly with instrumentation for NUnit 2.5.5 using nunit-console.exe. Although the NUnit console runner ran my test successfully via the generic VSTest, I was never able to generate code coverage results.

Although the test features of VS2010 sound great when reading about them, the implementation seems overly complex and very heavy. I've been using NUnit for years with great success and very low friction.

I decided to try out JetBrains dotCover beta and within 5 minutes I downloaded it, installed it, and was able to configure and use it very successfully. It integrates seemlessly into ReSharper 5 and just worked for my NUnit tests. The code highlighting is a great visual tool to quickly locate blocks of code that were not executed by the tests.

Wade
  • 356
  • 2
  • 7
2

You need to turn off shadowcopy in NUnit to get it to work. To do so, use the /noshadow flag.

e.g. C:\Program Files (x86)\NUnit 2.6\bin\nunit-console-x86.exe MyLibrary.dll /noshadow

Scott McKay
  • 190
  • 1
  • 8
John
  • 21
  • 1
2

I tried every method I found on Google to enable that and then decided to give TestDriven.NET a try. It worked brilliantly: right-clik on the solution, test with coverage and done. Seriously, it couldn't be any easier.

Bertrand Le Roy
  • 17,731
  • 2
  • 27
  • 33
2

I had the same problem and I found out a way to solve it. The project dll shown in "Code Coverage Detail" dialog is the wrong assembly to be instrumented and that's way is giving empty results. So you need to actually get that dll from another folder, it's in the unit test project folder! Since you added as a reference to your unit test project it will be there. So, for example, in my case I created a simple project calced Calculator and then created a unit test project called TestCalculator and added Calculator.dll as a reference in the unit test project. So in the "Code Coverage Detail" dialog I click on "Add Assembly" and I select Calculator.dll from the "..\TestCalculator\bin\Debug" folder. In that way it will work.

lili
  • 21
  • 1
1

To my knowledge it doesn't. Our TestMatrix tool does though.

sergeb
  • 2,484
  • 3
  • 22
  • 20
  • Thanks for commenting. +1. Don't know if I can trust this without a source if you're actually selling a competing product :-P So I'm gonna leave this hanging for a while longer.. – stiank81 May 04 '10 at 07:09
  • This is fair. You don't have to trust me - download a trial of VS2010 and see it yourself :) And we are not competing with VS - we are offering complementary products to enhance it... BTW, I just tried built-in to run dasBlog (uses NUnit) tests in VS2010 Premium - it doesn't see the test. VS2010 Trial download is here - http://msdn.microsoft.com/en-us/vstudio/bb984878.aspx – sergeb May 04 '10 at 08:28
1

Theoretically, it should. There is even documentation for it on MSDN. Basically, you need to set up a "generic test project" which wraps the NUnit test. In practice, I've been trying to get it to work for almost three hours now and it still doesn't. The assembly is instrumented, but there are no coverage results.

I always get this message: "Empty results generated: none of the instrumented binary was used. Look at test run details for any instrumentation problems.", but no problems are reported in that file.

Also, a VSPerf error in the event log pops up. I submitted this as a bug to Microsoft. However, this might be related to my machine. Since I submitted that bug just now, I don't know yet whether others can reproduce the issue or not.

mnemosyn
  • 45,391
  • 6
  • 76
  • 82
  • Sounds frustrating.. I didn't try myself yet, but it sounds like it should be possible then. Please shout out if you figure it out! – stiank81 May 06 '10 at 11:09
  • I tried to set this up and got nunit running instrumented assemblies, but code coverage data is missing. Upvoted your bug on MS site. – LeffeBrune Jul 15 '11 at 18:32
  • Make sure you use x86 runner for your tests (nunit-console-x86.exe). – LeffeBrune Jul 15 '11 at 18:56
0

Many of the answers in this thread helped me a lot, but it's hell to get all the little details from multiple answers.

I compiled a list of 6 hints that together with the marked answer by Scott Weinstein did the trick for me: Running NUnit tests in Visual Studio 2010 with code coverage

Although I don't deserve upvotes on both answers, just spare me downvotes for copying:

  • Scott Weinstein
  • John and Scott McKay
  • lili
  • mnemosyn

Upvote them!

Community
  • 1
  • 1
No answer
  • 907
  • 1
  • 9
  • 10