Using FsCheck with NUnit: receiving exception on using Arbitrary types (or: how to use Arbitrary types with attributes)

Question

In my previous question Kurt pointed me to this code of FsCheck about setting the Arbitrary type.

I have the following Arbitrary (disclaimer: I have no idea what I am doing..., still finding FsCheck notoriously hard to understand but I'm dead set on getting it to work), which in itself is a simplified version of something I created earlier:

type MyArb() =
    inherit Arbitrary<DoNotSize<int64>>()
        override x.Generator = Arb.Default.DoNotSizeInt64().Generator

And I use it as instructed:

[<Property(Verbose = true, Arbitrary= [| typeof<MyArb> |])>]
static member  MultiplyIdentity (x: int64) = x * 1L = x

This gives me a (somewhat hopeful) error message that I'm missing something:

 System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation.
  ----> System.Exception : No instances found on type Tests.Arithmetic.MyArb. Check that the type is public and has public static members with the right signature.
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at FsCheck.Runner.checkMethod(Config config, MethodInfo m, FSharpOption`1 target) in C:\Users\Kurt\Projects\FsCheck\FsCheck\src\FsCheck\Runner.fs:line 318
   at FsCheck.NUnit.Addin.FsCheckTestMethod.runTestMethod(TestResult testResult) in C:\Users\Kurt\Projects\FsCheck\FsCheck\src\FsCheck.NUnit.Addin\FsCheckTestMethod.fs:line 100

Looking back at that Github code I see two Atrbitrary classes but neither with any inheritance and they both have different static members.

How can I create a random-number generator and assign it as an Arbitrary statically to my NUnit tests?

Answer 1

The type you provide in the Property.Arbitrary parameter should have static members (possibly several) of type Arb. As in the code you linked:

type TestArbitrary2 =
   static member NegativeDouble() =
       Arb.Default.Float()
       |> Arb.mapFilter (abs >> ((-) 0.0)) (fun t -> t <= 0.0)

Applying this to your code, it should look like this:

 type MyArb() =
    static member m() = Arb.Default.DoNotSizeInt64()

---

The meaning of the Property.Arbitrary parameter is not "an implementation of Arbitrary", but rather "a bucket of typeclass implementations".

You see, the original Haskell implementation of QuickCheck relies on typeclasses to provide values of different types. In order for a particular type to be "quick-checkable", there needs to be an instance of the 'Arbitrary' class defined for that type (for example, here are instances for all basic types).

Since F# doesn't support type classes as such, FsCheck has to fake it, and this is the scheme used there: each type class instance is represented by a static member that returns the function table. For example, if we wanted to simulate the Eq typeclass, we'd define it something like this:

type Eq<'a> = { eq: 'a -> 'a -> bool; neq: 'a -> 'a -> bool }

type EqInstances() =
   static member ForInt() : Eq<int> = 
      { eq = (=); neq = (<>) }

   static member ForMyCustomType() : Eq<MyCustomType> = 
      { eq = fun a b -> a.CompareTo(b) = 0
        neq = fun a b -> a.CompareTo(b) <> 0 }

But because you can't just scan all static member in all loaded assemblies (that would be prohibitively expensive), there is this little inconvenience of providing the type explicitly (as a bonus, it allows to control the visibility of "instances").

Answer 2

This question demonstrates clearly, IMO, why the Reflection-based API for FsCheck is less than ideal. I tend to avoid that API completely, so I'd instead write the OP property like this:

open FsCheck
open FsCheck.Xunit

[<Property>]
let MultiplyIdentity () =
    Arb.Default.DoNotSizeInt64 () |> Prop.forAll <| fun (DoNotSize x) -> x * 1L = x

As the open directives suggest, this uses FsCheck.Xunit instead of FsCheck.NUnit, but AFAIK, there's no difference in the way the API works.

The advantage of this approach is that it's type-safe and more lightweight, because you don't have to implement static classes every time you need to tweak FsCheck.

Answer 3

If you prefer the approach described by Mark Seemann, then you may also consider using plain-FsCheck and get rid of FsCheck.Xunit entirely:

module Tests

open FsCheck

let [<Xunit.Fact>] ``Multiply Identity (passing)`` () = 
    Arb.Default.DoNotSizeInt64 ()
    |> Prop.forAll
    <| fun (DoNotSize x) ->
        x * 1L = x
    |> Check.QuickThrowOnFailure

let [<Xunit.Fact>] ``Multiply Identity (failing)`` () = 
    Arb.Default.DoNotSizeInt64 ()
    |> Prop.forAll
    <| fun (DoNotSize x) ->
        x * 1L = -1L |@ sprintf "(%A should equal %A)" (x * 1L) x
    |> Check.QuickThrowOnFailure

xUnit.net testrunner output:

------ Test started: Assembly: Library1.dll ------

Test 'Tests.Multiply Identity (failing)' failed: System.Exception:
    Falsifiable, after 1 test (2 shrinks) (StdGen (2100552947,296238694)):

Label of failing property: (0L should equal 0L)
Original:
DoNotSize -23143L
Shrunk:
DoNotSize 0L

    at <StartupCode$FsCheck>.$Runner.get_throwingRunner@365-1.Invoke(String me..
    at <StartupCode$FsCheck>.$Runner.get_throwingRunner@355.FsCheck-IRunner-On..
    at FsCheck.Runner.check[a](Config config, a p)
    at FsCheck.Check.QuickThrowOnFailure[Testable](Testable property)
    C:\Users\Nikos\Desktop\Library1\Library1\Library1.fs(15,0): at Tests.Multi..

1 passed, 1 failed, 0 skipped, took 0.82 seconds (xUnit.net 2.1.0 build 3179).