How to check if an Exception is thrown by a method with xUnit and FsCheck in F#

Question

I'm doing the Diamond Kata in C# with tests writen in F# using xUnit and FsCheck, and I'm having some trouble when trying to check if an Exception is thrown in case of a invalid input by the user (any char that isn't an alphabet letter without any diacritics). Here's how the code looks like now:

The method being tested:

public static string Make(char letter)
{
    if (!Regex.IsMatch(letter.ToString(), @"[a-zA-Z]"))
    {
        throw new InvalidOperationException();
    }

    // code that makes the diamond
}

The test:

[<Property>]
let ``Diamond.Make must throw an InvalidOperationException if a character that isn't 
      an alphabet letter without any diacritics is given`` (letter : char) =
    (not (('A' <= letter && letter <= 'Z') || ('a' <= letter && letter <= 'z'))) ==> lazy 
        (Assert.Throws<InvalidOperationException>(fun () -> Diamond.Make letter |> ignore))

The problem with my approach is that the test says that no Exception is thrown, but when I run the application with the input shown by the test suite the exception is raised.

Here is the message given by the test suite (I have intentionally omitted the test name and stack trace):

Test Outcome:   Failed
Test Duration:  0:00:00,066
Result Message: 
FsCheck.Xunit.PropertyFailedException : 
Falsifiable, after 1 test (0 shrinks) (StdGen (1154779780,296216747)):
Original:
')'
---- Assert.Throws() Failure
Expected: typeof(System.InvalidOperationException)
Actual:   (No exception was thrown)

Although the test suite says that for the value ')' no exception is thrown, I did a manual test with it and the expected exception was indeed thrown.

How can I make sure that the Exception is caught by the test?

Please post the entire test failure message that you're getting. Also, please format the code so that scrolling isn't necessary. — Mark Seemann, Oct 14 '16 at 19:09
My guess is that FsCheck finds it easy to falsify that property. There are many `char` values that fall outside the range given here, e.g. `ü` and `ø`. These are still letters according to `Char.IsLetter`. — Mark Seemann, Oct 14 '16 at 19:13
Do you understand that you are testing a Char.IsLetter now ? — FoggyFinder, Oct 14 '16 at 19:33
@FoggyFinder I don't think that's fair. The test is whether a precondition is correctly implemented, and if I understand the problem correctly, it's not. — Mark Seemann, Oct 14 '16 at 19:43
Thank you both! The Char.IsLetter was indeed a poor choice for evaluating the input, but now I've updated the code with a Regex and the test still don't catch any exception. Also, I've updated the question with the new code and test error message. Any ideas? Thanks! — Liordino Neto, Oct 14 '16 at 19:46
I also made a manual test with the value that the test suite states that causes the error on it and it indeed give me the expected exception, as stated in the question after I updated it. — Liordino Neto, Oct 14 '16 at 19:47
I think I found part of the problem: I'm using AspectInjector to log any error on this method, and I think that it was masking the exception. Now I removed the aspect from the method and the error is different: System.Exception : No instances of class FsCheck.Testable+Testable`1[a] for type System.InvalidOperationException — Liordino Neto, Oct 14 '16 at 19:52
Try unquote and/or watch Mark show you in the unit testing in F# course at http://www.shareasale.com/r.cfm?u=1017843&b=611266&m=53701&afftrack=&urllink=www%2Epluralsight%2Ecom%2Fauthor%2Fmark%2Dseemann - works much better with the language that xUnit's .Assert (xUnit itself is great, just the asserts are best replaced) — Ruben Bartelink, Oct 15 '16 at 07:28

Antti Leppänen · Accepted Answer · 2016-10-15T06:46:56.467

5

I think the problem is that Assert.Throws returns exception of given type if it occurs. Just ignoring return value of Assert.Throws should help you.

let test (letter : char) =
    (not (('A' <= letter && letter <= 'Z') || ('a' <= letter && letter <= 'z'))) ==> 

    lazy
        Assert.Throws<InvalidOperationException>(fun () -> Diamond.Make letter |> ignore)
        |> ignore

edited Oct 15 '16 at 06:46

answered Oct 14 '16 at 20:54

Antti Leppänen

363
2
7

Properties don't *have* to return a Boolean value; they can also return `unit`, or `Property`. In order to return `unit`, piping the result of `Assert.Throws` into `ignore` ought to be enough. – Mark Seemann Oct 15 '16 at 02:33

How to check if an Exception is thrown by a method with xUnit and FsCheck in F#

1 Answers1