ScalaCheck specificy minimum successful tests for property

Question

I'm trying to make sure that my ScalaCheck property runs 500 times instead of the default 100 times. I'm having trouble configuring this though.

class BlockSpec extends Properties("BlockSpec") with BitcoinSLogger {

  val myParams = Parameters.default.withMinSuccessfulTests(500)
  override def overrideParameters(p: Test.Parameters) = myParams

  property("Serialization symmetry") =
  Prop.forAll(BlockchainElementsGenerator.block) { block =>
    logger.warn("Hex:" + block.hex)
    Block(block.hex) == block
  }
}

However when I actually run this test it only says 100 tests passed successfully

EDIT:

$ sbt
[info] Loading project definition from /home/chris/dev/bitcoins-core/project
[info] Set current project to bitcoin-s-core (in build file:/home/chris/dev/bitcoins-core/)
> test-only *BlockSpec*
[info] + BlockSpec.Serialization symmetry: OK, passed 100 tests.
[info] Elapsed time: 1 min 59.775 sec 
[info] ScalaCheck
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] ScalaTest
[info] Run completed in 2 minutes.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 123 s, completed Aug 1, 2016 11:36:17 AM
>

How do I actually pass this to my property?

i guess you call that from sbt? can you show us how you call this property? and maybe try with `property.check` from a REPL? — jopasserat, Aug 01 '16 at 16:32
Looks like when I run it from the console I get only 100 tests passed as well `scala> res1._2.check + OK, passed 100 tests.` — Chris Stewart, Aug 01 '16 at 16:41
That does work, but it doesn't really answer the original question -- there should be some way to pass these parameters to the class and run them with sbt — Chris Stewart, Aug 01 '16 at 17:08

jopasserat · Answer 1 · 2017-03-19T19:34:06.790

As far as I understand you can specify the test parameters at two levels and they don't seem to communicate.

The first option is within the property as you're trying to do:

import org.scalacheck.Properties
import org.scalacheck.Test.{ TestCallback, Parameters }
import org.scalacheck.Prop.{ forAll, BooleanOperators }
import org.scalacheck.Test

class TestFoo extends Properties("BlockSpec") {

  override def overrideParameters(p: Parameters) = 
    p.withMinSuccessfulTests(1000000)

  property("Serialization symmetry") = forAll { n: Int =>
    (n > 0) ==> (math.abs(n) == n)
  }

}

This will have no impact as long as you don't call .check on the property. Can be from the sbt shell or directly within the class.

Now if you want to impact the number of tests run when calling the sbt:test target, it seems you have to play with options build.sbt (taken from here):

name := "scalacheck-demo"

scalaVersion := "2.11.5"

libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.2" % "test"

testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-maxSize", "5", "-minSuccessfulTests", "33", "-workers", "1", "-verbosity", "1")

This doesn't seem to allow per-test or per-property settings? — nafg, Dec 05 '16 at 05:46

score 1 · Answer 2 · answered Aug 01 '16 at 18:50

1

There's definitely an easier way to achieve that than overriding any kind of global test config:

class SampleTest extends FlatSpec
  with Matchers with GeneratorDrivenPropertyChecks {

  it should "work for a basic scenario" in {
    // This will require 500 successful tests to succeed
    forAll(minSuccessful(500)) { (d: String) =>
      whenever (d.nonEmpty) {
        d.length shouldBe > 0
      }
    }
  }
}

answered Aug 01 '16 at 18:50

flavian

28,161
11
65
105

Looks like you are using ScalaTest? – Chris Stewart Aug 01 '16 at 20:56
Hi @ChrisStewart yes that was just an example, the DSL is not ScalaTest specific, the forAll side at least. – flavian Aug 01 '16 at 23:44
how does this work with an explicit generator given in the `Prop.forAll()` function? – Chris Stewart Aug 02 '16 at 00:16
@ChrisStewart Look at the method signature, I think there's an available overload for you that takes the generator as the first argument and the min passes as the next. – flavian Aug 11 '16 at 12:27
I don't see any way to do this with vanilla scalacheck sbt framework. – nafg Dec 05 '16 at 05:45

Aaron Mast · Answer 3 · 2023-02-24T20:48:59.153

0

It looks a little different for me using munit. Here is what worked for me,

import munit.ScalaCheckSuite
import munit.FunSuite

class FoobarTest extends FunSuite with ScalaCheckSuite {
  property(
    "Foo does the bar"
  ) {
    forAll { (intValue: Int) =>
      assert(intValue == intValue)
    }
  }.check(_.withMinSuccessfulTests(5000))
}

edited Feb 24 '23 at 20:48

answered Feb 24 '23 at 20:39

Aaron Mast

249
3
4

ScalaCheck specificy minimum successful tests for property

3 Answers3