2

This is all i'm looking for at a high level:

  1. The ability to run unit tests from within visual studio 2008
  2. Compatibility with some code coverage tool
  3. A unit testing framework with a row testing feature like MBUnit's RowTestAttribute
  4. Ability to integrate with Team City

I thought i had found this with xUnit.net 1.8, resharper 6.0, and dotCover 1.1.1. However, after investing almost a day with messing around with this stuff i've found these major issues with this setup:

  • Theory tests (row tests) are not fully supported in Resharper - more info
  • dotCover basically does not work with xUnit

I was successfully using MSTest with ReSharper to do everything i need but the row tests, but it seems that life could be much better with some of the more advanced testing frameworks.

I've started looking at Gallio, but I'm hoping that someone can save me some time and recommend a winning combination before I waste any more time exploring.

What combination have you used successfully?

jaminto
  • 3,895
  • 3
  • 32
  • 36
  • 1. and 2.: These tools are orthogonal in purpose and compatibility to each other, and your IDE. You can use any unit testing framework with any IDE, and your existing code-coverage tools (see docs on how to use it with manual testing). Now, if you're talking about *strong integration*, that is different. Please clarify your exact needs – Merlyn Morgan-Graham Aug 11 '11 at 04:01
  • I understand that one solution does not provide all of these features, and that's what makes it difficult. I would define *strong integration* as what i was getting with resharper and MSTest - with one click you can run your test and view coverage, then edit the code and repeat all without leaving VS. I updated my question with a link to MBUnit's RowTestAttribute to clarify. – jaminto Aug 11 '11 at 04:05
  • 1
    NUnit supports a lot of things via extensions, has existing third-party free and paid integration with VS 2008, is the most likely to be integrated with any other product (since it is one of the oldest and well known), and has several parameterized test case options (some probably similar to your row tests): http://www.nunit.org/index.php?p=parameterizedTests&r=2.5.10 . I haven't tried resharper more than a minute, and I haven't worked with MSTest since digging into parameterized tests, so can't comment on them :) HTH tho – Merlyn Morgan-Graham Aug 11 '11 at 04:08
  • Another way to look at this: how long do your tests take to run? If running *all* tests takes too long, there may be work-arounds. E.g. you can categorize them to reduce this time, or try to test smaller "units"/improve your code isolation/code testability. If you do these work-arounds, you might be able to live with poor-man's integration, and just get creative. E.g. change your execute/debug command line for your unit test projects, link a test-runner library into your test libraries (so they are directly executable), or make a custom tool link/keyboard shortcut. – Merlyn Morgan-Graham Aug 11 '11 at 04:57
  • I do such a combination with NUnit and VS's built-in coverage. In VS, I execute a "custom test" that deploys my app and NUnit test, which I built to use the NUnit GUI/command line runner to execute itself. It isn't the prettiest solution, but my tests are fast, and our budget is hard to negotiate :) – Merlyn Morgan-Graham Aug 11 '11 at 05:01

3 Answers3

2

I would suggest you to use NUnit:

  • You can use TestDriven.NET or ReSharper to have full integration with Visual Studio
  • PartCover, Sonar, NCover 1.5.8 free as code coverage tool
  • TestCase attribute to pass different set of parameters (ReSharper supports this attribute perfectly)
  • Team City support is built in

An example for TestCase attribute

 [Test]
 [TestCase(1)]
 [TestCase(2)]
 [TestCase(3)]
 public void TestMe(int param)
 {
    Assert.That(param > 0);
 }
sll
  • 61,540
  • 22
  • 104
  • 156
0
  1. Integration with IDE: Out of the box and free => MSTest. If you're willing to download and use, there is a plethora of test runners.. NUnit being the oldest. If you're willing to pay, you have resharper, test-driven.net and the other add-ins. Resharper has its own bundled in test runner - so that's why you see those niggling issues. Now if you want to run and debug, then your options would be cut down even more. Running is as easy as setting the program to debug as your unit test runner exe. Bottomline: Integration for anything other than MSTest, has to be paid for/downloaded.
  2. Compatibility with code coverage: Again out of the box and free => MSTest. Paid: NCover will integrate with any exe... so not a constraint (but costs a bit too much IMHO). There are others too cropping up now - with Jetbrains throwing their own variant, I'm assuming compatibility will be a given among their own products.
  3. RowTests : NUnit has this. Although I think it was borrowed from xUnit or MBUnit, I forget. MSTest supports this as a downloadable extension.
  4. Integrate with Team City : See their page: Seems to work with NUnit and MSTest only. I have personally checked it out with NUnit.. worked with minimal configuration.
Gishu
  • 134,492
  • 47
  • 225
  • 308
  • The problem w/ 2 is that the OP didn't really define what they meant (without strong knowledge of what the reshaper integration provides). Most code coverage tools let you do coverage for manual tests, which means 1. instrument the build (at build time or coverage time) 2. run some coverage daemon to enable coverage collection 3. run the tests. However, some tools may let you take historical coverage data and select tests to execute based off code changes. If the OP means that level of integration, then options may be drastically narrowed without a lot of glue-tool work – Merlyn Morgan-Graham Aug 11 '11 at 04:49
0

I strongly suggest xUnit.net

Please read why it was built and how does xUnit.net compare to other .NET testing frameworks.

Nikos Baxevanis
  • 10,868
  • 2
  • 46
  • 80