1

I like to work with unittests and I think they help me alot. I use dotcover to analyze my coverage but as you know code coverage is not everything but I think it's still a vital tool.

Right now I'm working on a project with really high complexity and very high demands for accuracy(its about money so people are quite picky) so this is a project were just 100% code coverage is not enough. I would want to know that it will work every time.

Question: How can I measure my unit tests path coverage to ensure that the application works as intended? (This will require the unittests to as many as the cyclomatic complexity)

I know that this will require quite a effort and resources but in this case it's worth it.

I work with c# and visual studio.

Mikael
  • 43
  • 4

2 Answers2

1

I don't know of one off the shelf, so you will likely have to build one custom. That will "require a lot of effort"; what is the application?

You obviously need a tool that enumerate paths through code, and can instrument each path with tracking data.

For all this, you need a full C# front end (parsing, name/type resolution, control flow graph construction), followed by code instrumentation.

You might be able to process MSIL files, which capture the result of the C# compiler (e.g., all that front end information). There are "vanilla" test coverage tools that operate on MSIL files and do instrumentation. A possible downside is the that paths inside MSIL code may not map one-to-one to logical paths in your source code, which might make any answer a bit hard to decipher.

Roslyn certainly provide parsing and name/type resolution. I have no idea if it constructs control flow graphs. It isn't clear (to me, at least) what Roslyn offers for source code modification.

Our DMS Software Reengineering Toolkit has some support for C#, but not yet control flow graph extraction. What is relevant is that DMS provides machinery to do this, which has been used to implement precise control flow extraction for C++11 programs, so (by design) it surely has the capability to do so for C#.

Ira Baxter
  • 93,541
  • 22
  • 172
  • 341
  • Hmm, thanks for the tips, but I would(ofcourse) have a finished product that I can use. But otherwise I will have to this the hard way. It feels like it's a product that could be quite handy to have in general... – Mikael Jul 31 '14 at 17:10
0

OpenCover does branch coverage and will also record which test was running when a branch point was hit, however we are talking IL branches which may not map 1:1 with your code (as @IraBaxter has already mentioned). It does the latter by recording a stream of sequence points as they are visited whilst that test is running. Currently OpenCover just aggregates the results but you could alter the code to dump the raw stream which you would then need to analyze and produce some visualization of the paths taken.

An alternative approach is to try and keep your CC as small as possible usually by ensuring your classes do not do too much, i.e a complex network of small objects which are easier to unit test. Using dependency injection (e.g AutoFac) this is much easier to assemble than before and with mocking tools (e.g. NSubstitute, Moq) this is much easier to do than you think.

However even 100% sequence and branch coverage you can still have defects because it us the quality of your tests that matter, I would actually concentrate on this first and use these tools to guide you in the right direction.

Shaun Wilde
  • 8,228
  • 4
  • 36
  • 56
  • The "sequence" points are really just "it executed *here*", right? To do path coverage, something needs to enumerate all the *paths* (which you could then light up with the sequence point samples). It is getting the paths that is hard. – Ira Baxter Jul 31 '14 at 20:36
  • Thanks for the tip, I'll check openCover out. I agree with you about low CC and DI. My application utilizes already DI to 100% and FakeItEasy along with reflection to lower the CC – Mikael Aug 01 '14 at 06:49