Generating ScalaCheck tests with Cucumber JVM - Generic Functions

Question

To avoid X & Y problems, a little background:

I'm trying to set up a web project where I'm going to be duplicating business logic server and client side, client obviously in Javascript and the server in Scala. I plan to write business logic in Cucumber so I can make sure the tests and functionality line up on both sides. Finally, I'd like to have a crack at bringing ScalaCheck and JSCheck into this, generated input data rather than specified.

Basically, the statements would work like this:

Given statements select add generators. When statements specify functions to act upon those values in sequence. Then statements take the input data and the final result data and run a property.

The objective is to make this sort of thing composable so you could specify several generators, a set of actions to run on each of them, and then a set of properties that would each get run on the inputs and result.

Done this already in Javascript (technically Coffeescript), and of course with a dynamic language is straightforward to do. Basically what I want to be able to do in my scala step definitions is this, excuse the arbitrary test data:

class CucumberSteps extends ScalaDsl with EN
                    with ShouldMatchers with QuickCheckCucumberSteps {
  Given("""^an list of integer between 0 and 100$""") {
    addGenerator(Gen.containerOf[List, Int](Gen.choose(0,100)))
  }
  Given("""^an list of random string int 500 and 700$""") {
    addGenerator(Gen.containerOf[List, Int](Gen.choose(500,700)))
  }

  When("""^we concatenate the two lists$""") {
    addAction {(l1: List[Int], l2: List[Int]) => l1 ::: l2 }
  }

  Then("""^then the size of the result should equal the sum of the input sizes$""") {
    runProperty { (inputs: (List[Int], List[Int]), result: (List[Int])) =>
      inputs._1.size + inputs._2.size == result._1.size
    }
  }
}

So the key thing I want to do is create a trait QuickCheckCucumberSteps that will be the API, implementing addGenerator, addAction and runProperty.

Here's what I've roughed out so far, and where I get stuck:

trait QuickCheckCucumberSteps extends ShouldMatchers {

  private var generators = ArrayBuffer[Gen[Any]]()
  private var actions = ArrayBuffer[""AnyFunction""]()

  def addGenerator(newGen: Gen[Any]): Unit =
    generators += newGen

  def addAction(newFun: => ""AnyFunction""): Unit =
    actions += newFun

  def buildPartialProp = {
    val li = generators
    generators.length match {
      case 1 => forAll(li(0))_
      case 2 => forAll(li(0), li(1))_
      case 3 => forAll(li(0), li(1), li(2))_
      case 4 => forAll(li(0), li(1), li(2), li(3))_
      case _ => forAll(li(0), li(1), li(2), li(3), li(4))_
    }
  }

  def runProperty(propertyFunc: => Any): Prop = {
    val partial = buildPartialProp
    val property = partial {

      ??? // Need a function that takes x number of generator inputs,
          // applies each action in sequence
          // and then applies the `propertyFunc` to the
          // inputs and results.
    }

    val result = Test.check(new Test.Parameters.Default {},
                            property)

    result.status match {
      case Passed => println("passed all tests")
      case Failed(a, l) => fail(format(pretty(result), "", "", 75))
      case _ => println("other cases")
    }
  }
}

My key issue is this, I want to have the commented block become a function that takes all the added actions, apply them in order and then run and return the result of the property function. Is this possible to express with Scala's type system, and if so, how do I get started? Happy to do reading and earn this one, but I need at least a way forward as I don't know how to express it at this point. Happy to drop in my Javascript code if what I'm trying to make here isn't clear.

score 0 · Answer 1 · answered Nov 25 '13 at 05:37

0

If I were you, I wouldn't put ScalaCheck generator code within your Cucumber Given/When/Then statements :). The ScalaCheck api calls are part of the "test rig" - so not under test. Try this (not compiled/tested):

class CucumberSteps extends ScalaDsl with EN with ShouldMatchers {
  forAll(Gen.containerOf[List, Int](Gen.choose(0,100)), 
         Gen.containerOf[List, Int](Gen.choose(500,700)))
        ((l1: List[Int], l2: List[Int]) => {
          var result: Int = 0
          Given(s"""^a list of integer between 0 and 100: $l1 $""") {   }
          Given(s"""^a list of integer between 0 and 100: $l2 $""") {   }
          When("""^we concatenate the two lists$""") { result = l1 ::: l2 }
          Then("""^the size of the result should equal the sum of the input sizes$""") {
            l1.size + l2.size == result.size }
         })
}

answered Nov 25 '13 at 05:37

Glen Best

22,769
3
58
74

If I were to do that it seems like there's no purpose for me to even bother with cucumber as a generator in the first place. I may as well just write it by hand at that point and not bother with the abstraction. I've tested this approach through hard coding and the property assertion is fine with Cucumber. The missing link is the abstraction I want to create by allowing a stack of functions to be composed into a property. – MalucoMarinero Nov 26 '13 at 00:06
Cuc's a framework, not a generator. (+ Use Case/Reqs/Test Code repository, status dashboard). BDD flow: Concept->Spec->Test Definition->Code->Verification->Validation. Cuc manages step 2 through (most of) 5. Spec can still be written in plain text, possibly by BA/PM (no need to consider ScalaCheck). Test definition adds code to Given, but places ScalaCheck data generation beforehand. Gives full benefits of BDD & test data generation. As for abstracting stack of test functions - not in Q? Any tool needs code/config definition - easy solution: invoke utility test fns (poss with cascading). – Glen Best Nov 26 '13 at 02:55

Generating ScalaCheck tests with Cucumber JVM - Generic Functions

1 Answers1