0

I have a set of specifications that need to be ran in an exact order for the last specification to be relevant. These specifications need to be dependent on each other because of a complex use case involving server resiliency (1 server going down, another coming up to continue).

In order to make this happen, I've named my specifications alphanumerically as "T101_Something1", "T102_Something2", ... "T109_LastSpec".

However, when running my tests with the command "gradle firefoxTest", the tests are ran in a non alphanumeric order causing the last specification to be irrelevant and always fail.

How is the execution order of Spock specifications determined when using gradle with the Geb framework?

Dave
  • 2,126
  • 1
  • 15
  • 18

2 Answers2

4

Gradle's Test task doesn't give any guarantees about the order in which test classes will be executed. (If Test#maxParallelForks is set to a value greater than 1, they may even get executed in parallel.) The best practice for Spock/Geb is to keep order-dependent tests in the same test class annotated with @Stepwise. If you must enforce order between test classes, perhaps look into JUnit 4 test suites, which can be used together with Spock. Another (probably less desirable) option would be to split test execution across multiple Test tasks, whose execution order can be controlled with dependsOn.

Peter Niederwieser
  • 121,412
  • 21
  • 324
  • 259
  • thank you for the quick replies! It's unfortunate that gradle doesn't provide a means of guaranteed execution order. One solution would be to combine all relevant features into one spec, but that would be one bloated spec. That will be a last resort. I'll post if I find a better option. Thanks again! – Dave Jun 27 '14 at 16:03
3

We had to create a custom JUnit test runner and run it as:

gradlew -DfirefoxTest.single=CustomJUnitSpecRunner firefoxTest

The CustomJUnitSpecRunner.groovy file:

package spec

import org.junit.runner.RunWith
import org.junit.runners.Suite
import spec.Spec1
import spec.Spec2
import spec.Spec3

@RunWith(Suite.class)
@Suite.SuiteClasses([Spec1.class, Spec2.class, Spec3.class])
class CustomJUnitSpecRunner {
}

This allows us to guarantee the execution order of our Spock Specifications every time they are ran.

Dave
  • 2,126
  • 1
  • 15
  • 18