12

What is the best practice recommendation for doing TDD with .NET 4.0 Code Contracts?

I suppose specifically, my question is that given that one point of TDD is to allow the code to be self documenting and the contract now provides a part of the documentation, should code contracts be tested in the same way as other code?

satnhak
  • 9,407
  • 5
  • 63
  • 81

5 Answers5

11

This depends on how you are using contracts and what kind of application you are developing.

First of all: You certainly don't want to test assertions and postconditions (Contract.Assert, Contract.Assume, Contract.Ensures and Contract.EnsuresOnThrow) separetly. I see no practical value in doing so - as they are already verified during runtime by the rewriter, you will find failures very fast even without tests.
However, in a well tested application, no postcondition or assert should fail - even on invalid inputs. Thus, if all your tests (even the ones testing handling of invalid data!) pass without a single postcondition/assertion-fail, your postconditions and assertions can be seen as "tested".
For this, you might want to handle the ContractFailed event using an "Assert.Fail" within your tests.

Now the "interesting" part are the preconditions:
Are you developing a library? Then, you should definitely test them if your time/budget allows this (it's worse not testing the actual logic).
Especially, if you are using the "Contract.Requires<E>" overload which will throw a specific exception on contract failures, you should test them like regular parameter validation using "if-throw"-constructs.

If you are not writing a library, I wouldn't say that testing preconditions is really necessary - they are no real business requirement but rather a helper for debugging.
And it can get really boring to write a unittest for every ArgumentNullException a method should throw if a parameter is null.
If you forget this validation code (meaning: The specific Contract.Requires) inside your method, you will probably forget the unit-test, too. Therefore, the additional value a parameter-validation-test brings to your (non-library-)code is very low to the connected value.

To sum it up: Don't test postconditions and asserts. Do test preconditions - but only on libraries (and maybe parts of your code which are used like libraries).

Matthias
  • 12,053
  • 4
  • 49
  • 91
  • "Don't test postconditions and asserts. Do test preconditions - but only on libraries (and maybe parts of your code which are used like libraries)." - that sounds like pretty decent advice: is that based on your own experience, or on received wisdom? – satnhak May 01 '11 at 20:36
  • @B Tyler: It's based on both my experience and discussion with other programmers. However, it's still just my opinion and not backed up by any studies or long-time research ;-) – Matthias May 02 '11 at 14:23
  • Hmm, I rather think you should test all pre-condtions for the simple reason that you may make a mistake in the precondition logic itself (e.g. mistyping a number or a string). The only way to know you haven't screwed up in this manner is to test it. But other than that, I agree with you about not needing to test post-conditions and asserts. – fourpastmidnight Feb 27 '14 at 00:43
4

I'm going to disagree with some other folks here. Contracts are NOT tests, they are assertions about the API's requirements and promises. They don't magically prove your code to be correct, they just provide you information at runtime when you violate the contract. I don't know about you but I'd hate to ship code that in some corner case failed to comply with the contracts and crashed with a contract assertion! Like any other behavior, the contracts should be unit tested. If you don't excercise the contracts (and code paths that indirectly excercise the contracts) you have no evidence of the codes validity. Code contracts and unit tests are not mutually exclusive concepts.

wekempf
  • 2,738
  • 15
  • 16
  • 1
    They are not *just* runtime information; static verification can also be performed that causes compile-time errors (e.g. when a method is called with a parameter that could be null but the contract requires that it is not). – TheXenocide Nov 13 '14 at 21:58
1

Tests are there to test code behaves as expected

You should NOT explictly write tests to exercise contract assertions.

However in TDD or when making a code change, running a unit-test may exercise code in a way that causes a contract fails - when this happens the test should fail and you want need to be able to then find the contract that failed quickly and easily, so you can correct the code.

So somehow you want to catch a contractexception if only to then do a Asser.Fail("Contract requirement not met")

This may be more what you are after How to Log error while using Code Contracts

IRQ Conflict
  • 55
  • 5
1

Great question. The simple answer is no. code contracts can take care of the superfluous tests that are not about the behavior of the system. If you really can get to 100% code coverage, you will need to take care of the isnull checks, etc. these checks don't need to be in your test suite. the added benefit is that these will be checked at compile time instead of waiting for tests to execute.

Hope this helps.

Adam Dymitruk
  • 124,556
  • 26
  • 146
  • 141
1

A useful tool to create unit tests in combination with Code Contracts is Pex. It analyses your code and generates basic unit tests for it. The great thing is that it recognizes and understands Code Contracts, and so adapts the testing code it generates.

If you have an MSDN Subscription then you can download Pex/Moles as a powertool, otherwise you can download it (not the most recent version) at http://research.microsoft.com/en-us/projects/pex/downloads.aspx.

Roy Dictus
  • 32,551
  • 8
  • 60
  • 76