Is there any way how to test contracts set using Code contracts?

Question

I would like to write unit test that verify that my method does not accept invalid arguments. Validity of arguments is checked using Code Contract's Contract.Requires call. Why would I want to test contracts? I consider my tests to be kind of method specification (that is actually idea taken from TDD) so by testing that the method fails for some arguments I specify that such arguments should not be used.

The problem is that since I started to use Code contracts I cannot test method contracts because I cannot access the exception that is thrown by Contract.Requires. I can catch generic Exception but that just is not nice... Is there recommended/supported way how to test contract set using Code Contracts?

Seems to me that Code Contracts does not really support unit testing...

EDIT: My test example (I am forced to catch generic exception)

[ExpectedException(typeof(Exception), AllowDerivedTypes = true)]
public void Compute_Throws_ForNullArgument()
{
    new ComputingService().Compute(null);
}

What testing framework are you using? can you show the tests you are trying to write? — Caleb, Feb 24 '14 at 19:15
If you use contracts, you haven't need of unit tests. Or you are looking for 100% code coverage? :) — Hamlet Hakobyan, Feb 24 '14 at 19:17
Are you planning to unit test every facility that is provided to you by the environment? If you're using contracts, concentrate your testing efforts on situations not dealt with by contracts. — Damien_The_Unbeliever, Feb 24 '14 at 19:22
@HamletHakobyan I am definitely not using 100% code coverage. Why do you think I don't need unit tests? Code Contracts does not verify the code as well is good unit tests do. We would all by using Code contracts instead of unit tests if they would. — Rasto, Feb 24 '14 at 19:26
No, but picking up from Hamlet, if you've written the contracts then you can assume that (once your actual code is reached) those contracts are satisfied. Someone needs to write a test that if a `Contract.Requires` test fails then an exception is thrown. That person is someone inside MS, not you. — Damien_The_Unbeliever, Feb 24 '14 at 19:32
@drasto: I probably wouldn't validate *trivial* contracts, in the same way that I don't check that type safety is honoured: I assume the *system* works. I'd probably only unit test complicated contracts, simultaneously considering whether perhaps those contracts are too complicated to start with :) — Jon Skeet, Feb 24 '14 at 19:33
Please, do not include information about a language used in a question title unless it wouldn't make sense without it. Tags serve this purpose. — Ondrej Janacek, Feb 24 '14 at 19:46
My comment is states that if you use contracts to validate corner cases you won't need validate them also by unit tests. — Hamlet Hakobyan, Feb 24 '14 at 19:47

score 3 · Answer 1 · answered Feb 24 '14 at 19:20

You can hook into the Contract.ContractFailed event.

http://msdn.microsoft.com/en-us/library/system.diagnostics.contracts.contract.contractfailed(v=vs.110).aspx

This will be raised before the exception is thrown. You can combine this with catching Exception to be pretty certain that it represented a contract failure

public void VerifyContract(Action action) { 
  bool failed = false;
  bool thrown = false;
  EventHandler e = (sender, e) => { failed = true; }
  Contract.ContractFailed += e;
  try { 
    action();
  } catch (Execption) {
    Assert.True(failed);
    thrown = true;
  } finally {
    Contract.ContractFailed -= e;
  }
  Assert.True(thrown);
}

Jon Skeet · Answer 2 · 2014-02-24T19:31:55.323

2

You can't explicitly catch the right exception type, but you could catch Exception and then check that it's a ContractException using reflection (rethrowing otherwise).

That would be ugly to do everywhere, but you just need to do it once:

public static void AssertContractFailure(Action action)
{
    try
    {
        action();
        Assert.Fail("Expected contract violation");
    }
    catch (Exception e)
    {
        if (...) // I can't remember offhand what you'd need to check
        {
            throw;
        }
    }
}

Then:

AssertContractFailure(() => SomeContractViolation(...));

Currently if you have that in a helper class you'd need to qualify the call everywhere, but as of C# 6 you'll hopefully be able to import it easily :)

edited Feb 24 '14 at 19:31

answered Feb 24 '14 at 19:18

Jon Skeet

1,421,763
867
9,128
9,194

+1 as good as usually. I just wish method like this was part of Code contracts... Or even better that they provide attribute, something like `ExpectedContractAtribute` that would be analogous to `ExpectedExceptionAtribute`. – Rasto Feb 24 '14 at 19:32
2

@drasto: I wouldn't advise using `ExpectedExceptionAttribute` anyway: it means an exception can be thrown *anywhere* in your test rather than just where you expect it. That's why I use `Assert.Throws`. – Jon Skeet Feb 24 '14 at 19:34
I use `ExpectedExceptionAtribute` to test very simple case like that one in my edit, so there is often only one place that can cause expected exception. I will consider your advice anyway. Can you have a look on [JaredPar's answer](http://stackoverflow.com/a/21996934/311865) and perhaps leave a comment? It seems to me that there are no significant advantages in his approach over yours - except maybe if the type of exception thrown by Code contracts is changed his solution will be still working. – Rasto Feb 24 '14 at 19:49
1

@drasto: Both will work. I find my approach a little bit simpler, but it does require that the bit in dots for checking that it's the right exception. – Jon Skeet Feb 24 '14 at 19:50

score 0 · Answer 3 · answered Feb 24 '14 at 19:17

If I recall correctly, contracts throw exceptions if the Requires construct fails, and if the Return construct fails. Therefore surely for unit-testing purposes you simply need to catch these exceptions and you'll know if the contract was violated or not.

Is there any way how to test contracts set using Code contracts?

3 Answers3