Scala: Acceptance Testing Order

Question

for reference: How make tests always run in same order in Scalatest?

I plan to test my application by calling controllers/routes and comparing the responses to my expected ones.

I do not want to mock my persistence layer, so I can test it too. My approach would now be to execute tests in order to reflect user actions. Example:

Test 1: User registers 
--> Test 2: (depends on a existing user) User creates profile 
--> Test 3: (depends on a user with existing profile) User changes profile

So to save time, I do not want to mock anything for Test 2 and Test 3 but instead just work on the same database all the time and use the data generated by preceding tests.

Is this approach ok and how would one specify the execution order in Specs2 or ScalaTest?

Answer 1

Having no dependencies between individual test suites is preferred for at least two reasons:

• Being concerned with the order in which the suites are executed makes the test execution harder to understand
• If suite A depends on suite B, changing something in suite B may break suite A, meaning it is harder to find the cause of a failing test.

Because of these drawbacks I would recommend you to properly setup your persistence layer at the beginning of each acceptance test; at the expense of execution time. Note that you can tag your tests and only execute your slow acceptance tests occasionally to not slow down your development cycles.

If you want to implement dependent tests in ScalaTest nonetheless, you can create a nested test suite as is suggested in the question you linked:

Assuming your persistence layer:

object Users {
  var users: List[User] = Nil

  def apply(i: Int): User = users(i)

  def register(user: User): Unit = users = user :: users

  def isEmpty: Boolean = users.isEmpty
}

class User(var profile: Option[Profile] = None) {
  def createProfile(): Unit = profile = Some(new Profile)
}

class Profile(var content: String = "") {
  def update(newContent: String): Unit = content = newContent
}

and your individual tests:

@DoNotDiscover
class Test1 extends FlatSpec with ShouldMatchers {
  "register" should "store a new user" in {
    Users.register(new User)

    Users should not be 'empty
  }
}

@DoNotDiscover
class Test2 extends FlatSpec with ShouldMatchers {
  "createProfile" should "create a new user profile" in {
    val user = Users(0)
    user.createProfile()

    user.profile shouldBe 'defined
  }
}

@DoNotDiscover
class Test3 extends FlatSpec with ShouldMatchers {
  "update" should "update the content of the profile" in {
    val newContent = "Test"
    val profile = Users(0).profile.get
    profile.update(newContent)

    profile.content shouldBe newContent
  }
}

you can nest them in an acceptance test suite:

class AcceptanceTests extends Suites(
  new Test1,
  new Test2,
  new Test3
) with SequentialNestedSuiteExecution

The @DoNotDiscover annotation is necessary to prevent the test runner from executing the nested tests separatly (as they are itself test suites). Mixing in the trait SequentialNestedSuiteExecution guarantees that the nested tests are executed in the given order.