How to run a specific number of tests in QuickCheck?

Question

I am trying to learn Haskell and specifically QuickCheck. While Haskell has a lot of information online I am struggling to create some random test with QuickCheck.

For example, I have the following script:

import Test.QuickCheck

whatAge :: Int -> Int -> Int -> Int -> Bool

whatAge age1 age2 age3 age4
  | age1 + age2 + age3 + age4 == 5 = True
  | otherwise = False

main = do
    verboseCheck  whatAge

When I run it shows:

*** Failed! Falsifiable (after 1 test): 
0
0
0
0

Fairly enough it showed a test on which the function was false.

What I would like to do though is to:

Generate 200 random tests even on failure (a.k.a even when the output of the whatAge function is false)

Be able to put a range on my function parameters, for example:

   x1 range from 1 to 30

   x2 range from 1 to 40

   x3 range from 1 to 50

   x4 range from 1 to 60

Be able to generate non-repeating tests

From my understanding nr 3 is not really possible with QuickCheck, for that I will have to use smallCheck but I am not sure about point 1 and 2.

The second item sounds like it should be possible, but I'm having a hard time understanding why you want to do the two other things, or what you even mean by those, so perhaps I'm misunderstanding the second item as well. Can you elaborate on your overall goal? — Mark Seemann, Jun 11 '19 at 17:29
@MarkSeemann what I would like to do is to create 200 random tests with quickCheck on the function ´whatAge´ no matter the outcome of the tests (False or True). Where each input of the function should be in a specified range. — Andi Domi, Jun 11 '19 at 17:34
How should "generate 200 random tests even on failure" interact with shrinking? — Daniel Wagner, Jun 11 '19 at 18:02
I think the main issue is that you're using QuickCheck for something it was not intended for. QuickCheck should be used to verify that your function holds some property, for example in this case `check a1 a2 a3 a4 = whatAge a1 a2 a3 a4 == (a1 +a2+a3+a4==5)` — Lorenzo, Jun 11 '19 at 18:13
Also on another note, in Haskell saying `if this then True else False` is literally the same as saying `this`. So in your case, your function could be simplified as `whatAge a1 a2 a3 a4 = a1+a2+a3+a4 == 5` — Lorenzo, Jun 11 '19 at 18:15
You can use [implication](http://hackage.haskell.org/package/QuickCheck-2.13.1/docs/Test-QuickCheck.html#v:-61--61--62-) for number 2, but it would be better to define a generator for the range instead of discarding lots of possible tests. You can't do #1 nor #3 I believe. — Thomas M. DuBuisson, Jun 11 '19 at 18:19

Daniel Wagner · Accepted Answer · 2019-06-11T18:45:51.947

For simple properties of your input, you can make a newtype with an appropriate Arbitrary instance that captures them. So:

{-# LANGUAGE AllowAmbiguousTypes #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}

import Data.Proxy
import GHC.TypeLits
import Test.QuickCheck

newtype Range (m :: Nat) (n :: Nat) a = Range { getVal :: a }
    deriving (Eq, Ord, Read, Show, Num, Real, Enum, Integral)

numVal :: forall n a. (KnownNat n, Num a) => a
numVal = fromInteger (natVal @n Proxy)

instance (KnownNat m, KnownNat n, Arbitrary a, Integral a) => Arbitrary (Range m n a) where
    arbitrary = fromInteger <$> choose (numVal @m, numVal @n)
    shrink hi = go (numVal @m) where
        go lo | lo == hi = [] | otherwise = lo : go ((lo+hi+1)`div`2) -- overflow? what's that? lmao

whatAge :: Range 1 30 Int -> Range 1 40 Int -> Range 1 50 Int -> Range 1 60 Int -> Bool
whatAge (Range age1) (Range age2) (Range age3) (Range age4)
    = age1 + age2 + age3 + age4 == 5

In ghci:

> verboseCheck whatAge
Failed:  
Range {getVal = 17}
Range {getVal = 29}
Range {getVal = 3}
Range {getVal = 16}

Failed:                                  
Range {getVal = 1}
Range {getVal = 29}
Range {getVal = 3}
Range {getVal = 16}

Failed:                                               
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 3}
Range {getVal = 16}

Failed:                                                
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 16}

Failed:                                                
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 1}

*** Failed! Falsifiable (after 1 test and 4 shrinks):  
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 1}
Range {getVal = 1}

For more complicated properties, where it's not clear to how to directly create a random value that satisfies the property, you may use QuickCheck's (==>) operator. For example, for range checks as above:

> verboseCheck (\x -> (1 <= x && x <= 30) ==> x*2 < 60)
Skipped (precondition false):
0

Passed:                
1

*** Failed! Falsifiable (after 33 tests):                  
30

To make exactly 200 tests, you could call quickCheckWith to make one test, 200 times; or you could directly generate test results by calling your property on arbitrary manually.

The value-level `==>` is the buried lede here. The `Range` type is certainly nice, but there’s an overkill number of moving parts here considering OP’s stated level with Haskell. — Jon Purdy, Jun 12 '19 at 06:38

How to run a specific number of tests in QuickCheck?

1 Answers1