Cucumber: background fails still exits with code 0

Question

I was wondering about something, cucumber returns an exit code 0 (which means "ok" as far as i know) when a Background fails.

Now failing steps shouldn't probably be in the Background (at best in a 'before'-hook i guess??). But does anybody know with what philosophy it returns this exit code? Is it a bug or a feature?

Appendix: A more concrete example: Lets say this code passes:

 Feature: Figuring out how Cucumber works

 As a developer
 I want to find out why cuccies fail, but my build doesnt
 In order to have more confidence in my build

 Background: logging in into the system
   Given I am logged in

 Scenario: creating a new test set
   When I do something
   Then I should see "you've done something"

It returns with exit code 0. Lets make it fail:

 Background: logging in into the system
   Given I am logged in

 Scenario: creating a new test set
   Then I should see "there's no way you see this"
   When I do something
   Then I should see "you've done something"

The output shows a failing step and it returns with exit code 1 When I move the failing step to the Background:

 Background: logging in into the system
   Given I am logged in
   Then I should see "there's no way you see this"

 Scenario: creating a new test set
   When I do something
   Then I should see "you've done something"

The output still shows it failed, but it returns with exit code 0

score 5 · Accepted Answer · answered Dec 22 '11 at 07:13

5

I've asked the Cucumber guys (their mailing list) and they agree its a bug

http://groups.google.com/group/cukes/browse_thread/thread/e56699f0fabfc75f

answered Dec 22 '11 at 07:13

Len

2,093
4
34
51

score 1 · Answer 2 · answered Dec 19 '11 at 14:50

1

Background is really like Before, with the only difference being, that it is run after Before. And since you're using Background/Before (in general) to set prerequisites for your Scenario(s) it wouldn't be very helpful, if the testing-process fails, which an error code different from 0 would suggest, just because you made a mistake in your prerequisites.

So my guess is, although i can't be sure, that this is a 'feature' and very much intended so.

answered Dec 19 '11 at 14:50

robustus

3,626
1
26
24

But when a prerequisite fails, the scenario can't even be executed. Therefore I would expect the test-process to fail (an error code of not 0). – Len Dec 19 '11 at 16:10
No. Because the test didn't fail. It "just" wasn't executed. And the return values represents if the test failed. – robustus Dec 19 '11 at 17:43
2

Thanks, I guess my question is answered, though I'm not satisfied by the design decision (if it is one): In my opinion is a test that is not executed (especially when asked to execute it) *worse* than a test that was executed (whatever the outcome is). I'll ask it on the Cucumber mailing list, wondering how the contributors will respond – Len Dec 19 '11 at 17:57

Cucumber: background fails still exits with code 0

2 Answers2