Programmatically skip a test in Spock

Question

How do I programmatically skip a test in the Spock framework? I know I can annotate a test with @Ignore to skip it, or use @IgnoreIf to skip tests based on environmental variables and the like. But is there a way to run arbitrary code that decides whether or not a test should run?

For example, let's say I have an integration test that has to connect to a third-party service's sandbox environment. Outages in the service's sandbox environment cause the test to fail. Assuming I've written a method canConnectToService that checks if the test will be able to connect to this service, how do I write a test that will be skipped if canConnectToService() returns false?

I don't understand the question. You **can** run arbitrary code in `@IgnoreIf`. Have you even tried? I know this is an old question, but what exactly was your problem? I see no need to use `Assume` if an on-board Spock feature known to yourself already does the same. — kriegaex, Mar 15 '18 at 01:05
@kriegaex but this closure does not have access to test variables. The problem is how to ignore test in the middle of the test, based on some condition, know after test starts. — Krzysztof Wolny, May 23 '18 at 17:41
The problem is not technical, it is your testing approach. The other tests should not be skipped but all get red, failing fast. Skipped tests are a smell and not reported as failed in your stats. I always teach my coachee teams that the best feature of agile development is transparency. So if you have a 3rd party server outage you want to see 3,000 integration tests failing in order to see what a fundamental impact this problem has. Imagine you have only one failure and 2,999 skipped subsequent tests or even just 3,000 skipped ones, but also 10 more failing tests. _(to be continued)_ — kriegaex, May 26 '18 at 04:29
How does the development team react? Probably they are going to treat the skipped tests as minor problems because the test stats still look good, fixing the other failing tests first. What they should do instead is contact the 3rd party and make sure the problem causing loss of quality information for 3,000 tests gets fixed there ASAP. The same applies to in-house 3rd party systems or own components which are unavailable for whatever reason. Skipping tests is information hiding. _(to be continued)_ — kriegaex, May 26 '18 at 04:32
And BTW, you can still use `@Stepwise` within a single integration spec in order to make Spock automatically skip the remaining methods in that spec. BTW2, `@Requires` or `@IgnoreIf` can easily call a static method in your test, checking or logging into the remote system if you absolutely insist in doing so. — kriegaex, May 26 '18 at 04:32

score 16 · Accepted Answer · answered Nov 20 '15 at 04:17

16

Use JUnit's Assume class. Specifically, you could write Assume.assumeTrue(canConnectToService()) at the beginning of your test to skip the test if the third party service is unavailable. This method will throw an AssumptionViolatedException if canConnectToService() returns false, and Spock ignores tests that are interrupted by an AssumptionViolatedException for JUnit compatibility (see this bug report).

answered Nov 20 '15 at 04:17

fzzfzzfzz

1,248
1
12
22

it seems it is not working anymore. The exception is thrown and the test is failing. – lepe Jul 12 '22 at 04:11
3

For Spock 2.x, use JUnit 5's Assumptions.assumeTrue ( https://junit.org/junit5/docs/5.0.0/api/org/junit/jupiter/api/Assumptions.html ) or throw a TestAbortedException ( https://ota4j-team.github.io/opentest4j/docs/1.0.0/api/org/opentest4j/TestAbortedException.html?is-external=true ), since Spock 2 is based on Junit 5 (see https://github.com/spockframework/spock/issues/1185 ). – Magnus Reftel Sep 02 '22 at 08:18

lepe · Answer 2 · 2022-07-12T07:10:16.543

There is another alternative (maybe it didn't exists before):

Using instance inside @Requires or @IgnoreIf:

Examples using inheritance, but not required:

abstract class BaseTest extends Specification {
    abstract boolean serviceIsOnline()
 
    @Requires({ instance.serviceIsOnline() })
    def "some test" () { .. }
}

SubSpecification:

class OnlineTest extends BaseTest {
    boolean serviceIsOnline() { 
       // Test connection, etc.
       return true
    }
}

class SkipTest extends BaseTest {
    boolean serviceIsOnline() { 
       return false
    }
}

Documentation

instance

The specification instance, if instance fields, shared fields, or instance methods are needed. If this property is used, the whole annotated element cannot be skipped up-front without executing fixtures, data providers and similar. Instead, the whole workflow is followed up to the feature method invocation, where then the closure is checked, and it is decided whether to abort the specific iteration or not.

As an extra, another way you can programmatically skip a test is using the where label:

class MyTest extends Specification {
    
    List getAvailableServices() {
        // You can test connections here or your conditions
        // to enable testing or not.

        return available
    }

    @Unroll
    def "Testing something"() {
        setup:
           URL url = serviceUrl.toURL()
        expect:
           assert url.text.contains("Hello")
        where:
           serviceUrl << availableServices
    }

}

Programmatically skip a test in Spock

2 Answers2