Scalatest or specs2 with multiple test cases

Question

In TestNg and Java, we can run multiple test cases using DataProvider, and this runs as separate tests, meaning execution of a test isn't stopped on failure. Is there an analogue for ScalaTest or Specs/Specs2?

The question is confusing. Please provide an example of what do you mean. — Daniel C. Sobral, Jul 26 '11 at 18:41

Eric · Accepted Answer · 2011-07-24T20:21:39.003

25

In both ScalaTest and specs2, it is easy to create test cases at run-time, in order to parameterize them with data. Here's an example with specs2:

   class BasketSpecification extends Specification {

     "a basket must contain fruits" >> {
       Seq(apple, banana, orange) foreach { fruit => 
         ("it contains: " + fruit) >> {
           basket must contain(fruit)
         }
       }
     }
   }

Then the output is:

 A basket must contain fruits
 + it contains: apple
 + it contains: banana
 + it contains: orange

Whereas the following specification:

   class BasketSpecification extends Specification {

     "a basket must contain fruits" >> {
       Seq(apple, cake, orange) foreach { fruit => 
         ("it contains: " + fruit) >> {
           basket must contain(fruit)
         }
       }
     }
   }

Will print out something like:

 A basket must contain fruits
 + it contains: apple
 x it contains: cake
   'basket' does not contain 'cake'
 + it contains: orange

edited Jul 24 '11 at 20:21

answered Jul 24 '11 at 08:28

Eric

15,494
38
61

Reread my question. Your method runs as ONE test, meaning that a failure in one case is a failure of the test. in TestNg, this would be run as three tests, and so the failure information is meaningful – user44242 Jul 24 '11 at 13:18
Sorry that wasn't clear from the specification only that it is actually creating 3 tests. I added the output to show that. – Eric Jul 24 '11 at 20:22
See for updated syntax: http://etorreborre.github.io/specs2/guide/org.specs2.guide.Matchers.html#With+sequences – Alex Dean Jan 13 '14 at 09:06
4

it seems every specs2 version changes how to test examples in a loop, here's the latest way: https://etorreborre.github.io/specs2/guide/SPECS2-3.2/org.specs2.guide.ForLoops.html – pathikrit Mar 29 '15 at 12:07
1

I have to upvote above comment. I'm a long time Specs2 user. But I am **extremely** unhappy how unintuitive and also difficult it is to run some tests over a collection. Every time I have to consult the documentation, and also every time I struggle with getting it to run. I see how above example must work, but if you find (in the docs) something like `((_: Int) must beGreaterThan ...),foreach(...)`, and your usecase is like `((x: Int) => 5 must beGreaterThan(x)).foreach(...)` it doesn't work. But it's unclear why. – ziggystar Jan 19 '18 at 10:19

score 12 · Answer 2 · answered Jul 24 '11 at 18:53

That concept is called "shared tests" in ScalaTest, because the same test code is being "shared" by multiple fixtures, where "fixtures" are the "data" in TestNG's DataProvider approach. There's a way to do this for each style trait in ScalaTest that expresses tests as functions. Here's an example for WordSpec:

http://www.scalatest.org/scaladoc-1.6.1/#org.scalatest.WordSpec@SharedTests

You can alternatively just use a for loop to register the same test code for different data points. This came up in an email discussion that's here:

http://groups.google.com/group/scalatest-users/browse_thread/thread/7337628407b48064#

The for loop code in that case looked like:

  for (browser <- List("IE", "Chrome", "Firefox")) { 
    test(browser + ": test one") { driver => 
      info("Testing using " + driver) 
    } 
    test(browser + ": test two") { driver => 
      info("Testing using " + driver) 
    } 
    test(browser + ": test three") { driver => 
      info("Testing using " + driver) 
    } 
    test(browser + ": test four") { driver => 
      info("Testing using " + driver) 
    } 
    test(browser + ": test five") { driver => 
      info("Testing using " + driver) 
    } 
  } 
}

This actually registers 15 tests, five tests for each browser driver. This I believe is what you're after.

score 0 · Answer 3 · answered Jan 27 '20 at 13:34

ScalaTest offers Table-driven property checks Using this facility you can run a test for different inputs:

   import org.scalatest.prop.TableDrivenPropertyChecks._

val fractions =
  Table(
    ("n", "d"),  // First tuple defines column names
    (  1,   2),  // Subsequent tuples define the data
    ( -1,   2),
    (  1,  -2),
    ( -1,  -2),
    (  3,   1),
    ( -3,   1),
    ( -3,   0),
    (  3,  -1),
    (  3,  Integer.MIN_VALUE),
    (Integer.MIN_VALUE, 3),
    ( -3,  -1)
  )
/*------------------------------------------------*/
import org.scalatest.matchers.ShouldMatchers._

forAll (fractions) { (n: Int, d: Int) =>

  whenever (d != 0 && d != Integer.MIN_VALUE
      && n != Integer.MIN_VALUE) {

    val f = new Fraction(n, d)

    if (n < 0 && d < 0 || n > 0 && d > 0)
      f.numer should be > 0
    else if (n != 0)
      f.numer should be < 0
    else
      f.numer should be === 0

    f.denom should be > 0
  }
}

Scalatest or specs2 with multiple test cases

3 Answers3