5

I'm trying to implement a simple property check but Scalacheck is ignoring my generators. What I'm doing wrong here?

object AlgorithmTest extends Properties("Algorithm") {
  property("Test") = forAll (Gen.choose(0,10)) (n => n>=0 & n<10)
}

and this is the result in SBT

[info] ! Algorithm.Test: Falsified after 12 passed tests. [info] >
ARG_0: -1 [error] Failed: : Total 1, Failed 1, Errors 0, Passed 0,
Skipped 0
tonicebrian
  • 4,715
  • 5
  • 41
  • 65
  • The issue has now been fixed (for built-in generators) via the use of `suchThat` postconditions - see https://github.com/rickynils/scalacheck/issues/8 – DNA Jul 23 '14 at 15:15

1 Answers1

5

It looks like the Shrink instance which is passed to the forAll method is not using the generator when searching for smaller counter-examples. If you change your property to:

property("Test") = Prop.forAllNoShrink(Gen.choose(1, 10)) (n => n >= 0 && n < 10)

Then it should properly fail with:

[info] ! Algorithm.Test: Falsified after 7 passed tests.
[info] > ARG_0: 10
[error] Failed: : Total 1, Failed 1, Errors 0, Passed 0, Skipped 0

One way to visualize the Shrink values is to use the Prop.collect method:

property("Test") = Prop.forAll(Gen.choose(1, 10)) { n =>
  Prop.collect(n) { n >= 0 && n < 10 }
}

Then the collected values look like:

[info] ! Algorithm.Test: Falsified after 40 passed tests.
[info] > ARG_0: -1
[info] > Collected test data: 
[info] 17% 3
[info] 17% 1
[info] 15% 6
[info] 12% 9
[info] 10% 2
[info] 10% 5
[info] 7% 4
[info] 7% 8
[info] 2% -1
[info] 2% 7

Where you can see that -1 has been used during the shrinking process.

Eric
  • 15,494
  • 38
  • 61
  • 2
    But this is not the documented behaviour. If I use a generator between 1 and 10 all the values must be in the range [1,10[ . Never should appear a -1 – tonicebrian May 31 '12 at 13:43
  • 2
    @ancechu The `-1` appears _after_ the test failed, as ScalaCheck tries to find simpler input that fails the test -- that's the "shrinking". That phase doesn't respect the generation process -- whether that's a bug or an intrinsic limitation, I don't know. – Daniel C. Sobral May 31 '12 at 18:51
  • I should have searched yesterday but this issue is actually discussed [here](https://github.com/rickynils/scalacheck/issues/18) – Eric May 31 '12 at 20:30
  • The issue has now been fixed (for built-in generators) via the use of `suchThat` postconditions - see https://github.com/rickynils/scalacheck/issues/8 – DNA Jul 23 '14 at 15:14