2

I'm (learning how to) writing and running Geb tests in Intellij. How do I configure Geb so that it runs all my tests rather than stop at the first fail and leaving the rest not run?

Szuturon
  • 931
  • 4
  • 17
  • 28
  • What do you mean by run all the tests? All tests in one class? All tests in a suite? Which test framework are you using with Geb? How exactly are you running your tests? I've never seen tests being skipped after a failure unless you use Spock and `@Stepwise` – erdi Feb 18 '14 at 18:43
  • I am using geb-spock. When ever a test fails in my Spec, it exits the run and reports the failure. I am also using @Stepwise. In fact my entire project was cloned off of geb's Github. I'm using their Maven example. My code has just been plugged in as I am learning how to use Geb. – Szuturon Feb 18 '14 at 21:55

2 Answers2

3

When using Spock's @Stepwise all feature methods are run in the order of their declaration in the spec and if one feature method fails then all the following feature methods are being skipped.

ak1ra
  • 383
  • 4
  • 8
erdi
  • 6,944
  • 18
  • 28
  • I'm starting to get the sense I've been looking at Geb the wrong way from a QA point of view. I figured each test case to be run was its own method (should do blahblahblah block). I'm starting to think now that perhaps each test case is its own Spec instead? – Szuturon Feb 20 '14 at 01:01
  • 1
    Having multiple test cases (feature methods) in one Specification class is the most common scenario. – erdi Feb 20 '14 at 14:20
  • Huh. I guess that means I ought to be running login and logout steps for each individual feature method? I'm thinking with that setup, I can run the tests without @Stepwise, allowing the spec to finish running all its test cases. Currently I'm just having the login step run once at the beginning. – Szuturon Feb 20 '14 at 14:41
  • 1
    It depends on how you approach your testing. As a dev I prefer for my tests to be standalone so that I'm able to run any of them on its own. This means that I usually have infrustructure ([Groovy Remote Control](http://groovy.codehaus.org/modules/remote/)) that allows me to setup a logged-in session without having to login through login screens and I have separate tests for checking login screens - kind of grey box testing. As a QA you probably want to have "stories" so @Stepwise annotated Specs with several feature methods in them and logging the user in as part of setupSpec() make sense.. – erdi Feb 20 '14 at 15:03
  • As a QA I need to be testing this in such a way a user would be interacting with an application. I suppose it's fine if I make Geb login and logout for each test case, but when those steps are being run thousands of times, I feel like I'm burning time when I don't need to. Is there no way have my tests run in order as well as continue running despite of failures? – Szuturon Feb 20 '14 at 15:23
  • 1
    At the moment Spock runs tests inside of a spec in the order of declaration if if it's not `@Stepwise` annotated but it's not guaranteed to stay that way in the future versions of Spock. – erdi Feb 20 '14 at 15:50
  • 1
    Note that if you don't annotate your Spec with @Stepwise the cookies and thus session will be cleared by Geb between every feature method. You can always turn it of using [this config option](http://www.gebish.org/manual/current/configuration.html#auto_clearing_cookies) but that's a way to end up with some serious test bleed which will lead to flakiness and strange behaviour on failures... – erdi Feb 20 '14 at 15:53
  • Is there any way to configure stepwise to not fail the entire testsuite after a single test failure? – Jim Chertkov Jun 22 '15 at 09:49
1

I created a @StepThrough annotation by subclassing @Stepwise and taking out the line of code that fails the entire testsuite after a single failure. You can grab the code here

Community
  • 1
  • 1
Jim Chertkov
  • 1,199
  • 12
  • 20