0

I attempted to test the ping-pong example (from activator's minimal-akka-scala-seed) using both scalatest, and specs2 (I wrote the specs2 tests myself; see below).

The scalatest always passes. However, the specs2 tests do not pass all the time (i.e. if I rerun the same tests several times). When it fails, it has the following output:

   [info] PingPongActorSpecs2
   [info] 
   [info] A Ping actor in specs2 should should
   [error]   ! send back a ping on a pong
   [error]    assertion failed: expected PingMessage(ping), found PongMessage(pong) (TestKit.scala:339)
   [error] akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:339)
   [error] akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:315)
   [error] akka.testkit.TestKit.expectMsg(TestKit.scala:718)
   [error] com.example.PingPongActorSpecs2$$anonfun$1$$anonfun$apply$1$$anon$1$$anonfun$2.apply(PingPongActor2Spec.scala:27)
   [error] com.example.PingPongActorSpecs2$$anonfun$1$$anonfun$apply$1$$anon$1$$anonfun$2.apply(PingPongActor2Spec.scala:25)
   [error] akka.testkit.TestKitBase$class.within(TestKit.scala:296)
   [error] akka.testkit.TestKit.within(TestKit.scala:718)
   [error] akka.testkit.TestKitBase$class.within(TestKit.scala:310)
   [error] akka.testkit.TestKit.within(TestKit.scala:718)
   [error] com.example.PingPongActorSpecs2$$anonfun$1$$anonfun$apply$1$$anon$1.<init>(PingPongActor2Spec.scala:25)
   [error] com.example.PingPongActorSpecs2$$anonfun$1$$anonfun$apply$1.apply(PingPongActor2Spec.scala:23)
   [error] com.example.PingPongActorSpecs2$$anonfun$1$$anonfun$apply$1.apply(PingPongActor2Spec.scala:23)
   [info] 
   [info] A Pong actor in specs2 should should
   [error]   ! send back a pong on a ping
   [error]    assertion failed: expected PongMessage(pong), found PingMessage(ping) (TestKit.scala:339)
   [error] akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:339)
   [error] akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:315)
   [error] akka.testkit.TestKit.expectMsg(TestKit.scala:718)
   [error] com.example.PingPongActorSpecs2$$anonfun$3$$anonfun$apply$2$$anon$2$$anonfun$4.apply(PingPongActor2Spec.scala:37)
   [error] com.example.PingPongActorSpecs2$$anonfun$3$$anonfun$apply$2$$anon$2$$anonfun$4.apply(PingPongActor2Spec.scala:35)
   [error] akka.testkit.TestKitBase$class.within(TestKit.scala:296)
   [error] akka.testkit.TestKit.within(TestKit.scala:718)
   [error] akka.testkit.TestKitBase$class.within(TestKit.scala:310)
   [error] akka.testkit.TestKit.within(TestKit.scala:718)
   [error] com.example.PingPongActorSpecs2$$anonfun$3$$anonfun$apply$2$$anon$2.<init>(PingPongActor2Spec.scala:35)
   [error] com.example.PingPongActorSpecs2$$anonfun$3$$anonfun$apply$2.apply(PingPongActor2Spec.scala:33)
   [error] com.example.PingPongActorSpecs2$$anonfun$3$$anonfun$apply$2.apply(PingPongActor2Spec.scala:33)
   [info] 
   [info] 
   [info] Total for specification PingPongActorSpecs2
   [info] Finished in 19 ms
   [info] 3 examples, 0 failure, 2 errors

The specs2 tests I implemented are as follows (very similar to the scalatest in the test folder of the minimal-akka-scala-seed):

 package com.example

 import org.specs2.mutable.SpecificationLike
 import org.specs2.specification.AfterAll

 import akka.actor.{ Actor, ActorSystem, Props }
 import akka.testkit.{ TestKit, ImplicitSender }

 class PingPongActorSpecs2(_system: ActorSystem) extends TestKit(_system) 
   with ImplicitSender 
   with SpecificationLike 
   with AfterAll {
   import scala.concurrent.duration._

   def this() = this(ActorSystem("MySpec"))

   override def afterAll {
     TestKit.shutdownActorSystem(system)
   }

   lazy val waitTime = 1 seconds

   "A Ping actor in specs2 should" >> {
     "send back a ping on a pong" >> {
        val pingActor = system.actorOf(PingActor.props)
        within (waitTime) {
           pingActor ! PongActor.PongMessage("pong")
           expectMsg(PingActor.PingMessage("ping"))
           true
        }
     }
   }

   "A Pong actor in specs2 should" >> {
    "send back a pong on a ping" >> {
      val pongActor = system.actorOf(PongActor.props)
        within (waitTime) {
          pongActor ! PingActor.PingMessage("ping")
          expectMsg(PongActor.PongMessage("pong"))
          true
        }
     }
   }
 }

My library dependencies are as follows:

 scalaVersion := "2.11.6"

 lazy val akkaVersion = "2.3.13"

 lazy val scalaTestVersion = "2.2.4"

 lazy val specs2Version = "3.6"

 libraryDependencies ++= Seq(
   "com.typesafe.akka" %% "akka-actor" % akkaVersion,
   "com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test",
   "org.scalatest" %% "scalatest" % scalaTestVersion % "test",
   "org.specs2" %% "specs2-core" % specs2Version
 )

Does anyone know where I am screwing up in my specs2 tests?

Sunil Sandhu
  • 385
  • 4
  • 13

1 Answers1

1

Specs2 specifications execute examples concurrently so it is possible that your 2 examples interact with each other. You can either create one system per example (expensive) or use the sequential argument to have your tests being executed sequentially:

class PingPongActorSpecs2(_system: ActorSystem) extends TestKit(_system) 
   with ImplicitSender 
   with SpecificationLike 
   with AfterAll {

  import scala.concurrent.duration._
  sequential

  ... 
}
Eric
  • 15,494
  • 38
  • 61
  • Thanks Eric. That works. Ran with and without the sequential argument 20 times each. With the sequential argument, it passes 20/20. Without the sequential argument, it passes 15/20. – Sunil Sandhu Jan 08 '16 at 03:20
  • I also found from the [ScalaTest doc](http://doc.scalatest.org/2.2.4/index.html#org.scalatest.ParallelTestExecution) that _ScalaTest's normal approach for running suites of tests in parallel is to run different suites in parallel, but the tests of any one suite sequentially._ So that explains why the scalatest was passing all the time. – Sunil Sandhu Jan 08 '16 at 03:34
  • Yes this is indeed one major difference between the default settings of ScalaTest and specs2. The reason for being concurrent in default in specs2 is to encourage tests isolation at first. – Eric Jan 08 '16 at 03:37