4

Java Spring Cucumber, how can I continue the scenarios, and don't stop upon failure. As I run a Feature file, If a Scenario Failed, all other scenarios are ignored, which isn't the desired behaviour. I don't want to use assertionError, I would like to run a scenario, if it passed I'll log it as success, else log it as failure.

Example: an unstable module that i want to test, i would like to run 100 tests, and if 99 tests will pass, that will answer my demands. but say the first scenatio failed, so 0 out of 100 will pass, which isn't good for me

MichaelK
  • 189
  • 2
  • 11
  • Are you certain that those next test will be reliable? I mean, when something failed at beginning. Then this problem may propagate on next tests. If it's not. Then you should maybe create separate scenarios? http://stackoverflow.com/questions/15298521/continue-running-cucumber-steps-after-a-failure – Nickname0222022022 Aug 22 '16 at 11:50
  • I'm building tests such that many scenarios are tested, yet I don't want my Tests to be ignored upon failure, i perfer them to continue and fail, rather then ignored – MichaelK Aug 22 '16 at 11:55
  • 1
    I'm not sure if I understand your problem correctly. Normally, when 1 scenario (1 test) fails, other scenarios are still executed by cucumber. Are you saying its not? perhaps show us a bit of your code then? – Len Aug 22 '16 at 14:07
  • yes, the code is useless since this is a theoretical question, i'm asking if this kind of scenario exists. but you got me correctly, if a test has failed then the other tests run, but their status is "Test ignored." other scenarios run well. i want to allow my test to "pass" even though 1 test has failed – MichaelK Aug 22 '16 at 14:10
  • 1
    Are you talking about steps in a Scenario? If a step fails then the rest of the steps are skipped. As @SirLenz0rlot pointed out one scenario failing has no effect on the next scenario. – Grasshopper Aug 22 '16 at 14:22

2 Answers2

0

Short answer: no.

I think by 'tests' you mean what cucumber refers to as 'steps'.

Cucumber will not ignore other scenarios when one scenario fails, but it will stop the current scenario.

In any case I can think of, this is exactly what you want.

As a scenario specifies a path through your software. If you're not on that path anymore (a failing step) then it doesn't make sense to try to follow that path.

'General' tools, such as eclipses Junit runner may make this look a bit odd, by showing steps as a test, and therefore mark remaining steps as skipped tests. But steps != tests.

Len
  • 2,093
  • 4
  • 34
  • 51
0

If you want to make sure that all the steps in your scenario are executed you will have to use something like an ErrorCollector class as in junit. But this will only take care of AssertionError anything else and you will have skipped steps.

This is implemented with @Rules annotation which unfortunately cucumber runner does not recognize. You will have to manually instantiate it and call the verify method. Add each assertion you want to accomplish using the checkThat() method. In the last step of your scenario call verify() method.

Grasshopper
  • 8,908
  • 2
  • 17
  • 32