0

Using Scalatest is there a way to mark a test as "optional". What I mean is running it anyway, but if it fails do not mark the whole test execution as failed (Return 0 instead of 1 to the shell).

I am aware that ignore exists as a replacement for in, but that does not exactly do what I am willing to do.

The reason I want to do this is that I have a project in which some tests might run in some networks and in some other fail.

Is this possible?

Camilo Sampedro
  • 1,306
  • 1
  • 19
  • 32
  • 2
    I'm not sure if such option exists, but I think you better focus on solving the issue, or pass the test if the bad network is detected. – Maroun May 09 '18 at 14:25
  • Another option is to use the `cancel` if bad network is detected, but it just aborts everything, is there a way to abort just a single test given a condition? – Camilo Sampedro May 09 '18 at 14:30
  • 1
    I think a simple `return` would do that. – Maroun May 09 '18 at 14:31
  • Also, possible duplicate of https://stackoverflow.com/questions/32460295/how-to-programmatically-ignore-skip-tests-with-scalatest – tilde May 11 '18 at 18:07

1 Answers1

0

You can use tags to define groups of tests. You could either whitelist or blacklist tests. If you want to whitelist tests, define a tag for each environment, and if a test should pass in that environment, give it that tag. Then, in each environment, include tests with the tag for that environment with -n. If you want to blacklist tests, tag tests that should fail in each environment, and then exclude tests with that tag using -l. Filtering tests is done with runner configuration.

Here is an example for blacklisting tests that require an external service.

object ServiceTest extends Tag("com.mycompany.tags.ServiceTest")

class MyTest extends WordSpec {
  "a foo" should {
    "connect to the service" taggedAs ServiceTest in {
      // ...
    }

    "do something without the service" in {
      // ...
    }
  }
}

You could also use cancel() to abort a test. I'm not sure what you mean by "aborts everything", only the current test case is canceled. But I think it is better to set expectations for tests to pass or fail in a given environment, and if they don't, investigate. Otherwise, it is too easy to ignore canceled tests if the network is unavailable and there is a real problem, say the url changed.

class MyTest extends WordSpec {
  "a foo" should {
    "connect to the service" in {
      if (serviceUnavailable()) cancel("Can't connect to service.")

      // ...
    }

    "do something without the network" in {
      // ...
    }
  }

  def serviceUnavailable(): Boolean = ???
}
tilde
  • 848
  • 8
  • 19