4

If you're developing application using the code contracts, you may know, that this concept was introduced in Eiffel programming language.

I have become very confused after trying this concept in my C# application using System.Diagnostics.Contracts.

The main question for me is the next:

Are unit-tests really needed, if you have code contracts?

One of the major differences, that unit-test frameworks usually don't provide, is the possibility to call private methods (except MS fakes library with its shims). It's done, because of supporting composition & the idea, that private methods are covered by public method calls.

As for the code contracts, I can declare Contract.Requires, Contract.Ensures for private methods.

So, why do I need unit-testing, when I have code-contracts, which behavior is very similar.

Thanks

  • 5
    Are you able to write a code contract for every aspect that you'd ever want to test? And is the contracts system able to check (at build time) every one of those contracts? That sounds unlikely to me. – Jon Skeet Apr 12 '16 at 16:28
  • @JonSkeet I suppose, that if I want to have a product without bugs (at certain version) - I need to provide a very high % of code coverage. So, as for unit-testing (where I'm always trying to achieve 100%), I suppose try to do the same for the code-contracts. Also TDD-behavior (with each cycle R->G->R) supposes to develop the clean code for each aim individually, you can't switch to another stuff, when you didn't finish the previous one, do you? –  Apr 12 '16 at 16:33
  • 3
    I don't view code coverage as a particularly good proxy metric for code quality, to be honest. It's feasible to have 100% code coverage without really thinking about corner cases. – Jon Skeet Apr 12 '16 at 17:01
  • 3
    I am confused by the question. How do you know that you got the contracts right if you don't test them? – Eric Lippert Apr 12 '16 at 17:13
  • @Eric by that same thinking.. how do you know you get the unit test right if you don't unit test it? So its all just a way to confirm that it does what you intended it to do and you can use the test or contract to defend your code later, writing tests to cover all corner cases in my opinion its not cost effective. – DonO Apr 12 '16 at 17:54
  • Great question. I have asked interview candidates "how do you know that your test framework correctly tells you when a program fails?" and had candidates be unable to answer the question. I have also seen unit tests which had bugs in them that caused them to always pass no matter what the code under test did; you could have deleted the code under test and they'd still pass. Now, you mention cost effectiveness. I have seen bugs in programs that would have charged prices in dollars for items denominated in euros. Would you care to characterize the cost-benefit ratio of finding that bug? – Eric Lippert Apr 12 '16 at 18:02
  • @Eric I see where you going same thing if you are writing software that flies a plane than you can spend a ton more on testing however you still would not be 100% sure that everything is right. What i'm trying to say is that there should be a balance point based on the importance of the software and knowing that you can never get it 100% right. A code contract can be written to catch the euro dollar problem so why would you write a unit test also? Its not like if you write them both you are safer you just testing the same thing twice. – DonO Apr 12 '16 at 18:11
  • 3
    Why do I need to brush if I floss and use mouthwash? In all seriousness, this question has a few issues; 1) It assumes all situations that mandate the use of code contracts closely mirrors the reason for unit tests; 2) it assumes that unit tests have only one purpose; 3) it assumes that there will be one answer which will fit all. The answer of course, is that none of these are true, or all of them are true; it depends on the specifics of the project, its target audience, why the contracts are there, why the unit tests are there, and your goals. – George Stocker Apr 12 '16 at 18:21
  • @EricLippert: talking of the correctness of a test framework, there is a [question](http://stackoverflow.com/q/561485/1846281) about the correctness of unit tests. Would you like to make your valuable contribution to it? – Luca Cremonesi Apr 13 '16 at 15:47
  • @LucaCremonesi: There are 17 pretty good answers there already. I might add "consider running static analysis on your unit tests as well as your code". Analysis often produces a high false positive rate on unit tests because unit tests do things like deliberately pass bad data to make sure the exception is thrown. But if you can find, say, dead code via static analysis of a unit test, then odds are very good that the unit test is passing regardless of the correctness of the code under test! – Eric Lippert Apr 13 '16 at 15:51
  • This question reminds me of the famous quote of Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it." – Eric Lippert Apr 13 '16 at 15:56

2 Answers2

0

I would say no. By using code contracts you are defining what your code is supposed to do and checking that it is doing it. The unit test does the same thing for the most part so I believe it is redundant to the point that it is not cost effective to write both.

DonO
  • 1,030
  • 1
  • 13
  • 27
0

You surely need Unit testing. With code contracts, you can only have your static contract verification. There's much more you can do when running your code.

For example, say you are testing a class that depends on IConnectionProvider. What happens when your GetConnection throws? Code contracts won't help you with that.

Ideally you'd be testing the public methods of your class with different inputs, and verifying that it behaves as expected. This will help you find bugs, and in the long run, design better code.

Bruno Garcia
  • 6,029
  • 3
  • 25
  • 38