Asserting that a particular exception is thrown in Cucumber

Question

Scenario

I'm writing a library (no Ruby on Rails) for which I'd like to have very detailed Cucumber features. This especially includes describing errors/exceptions that should be thrown in various cases.

Example

The most intuitive way to write the Cucumber steps would probably be something like

When I do something unwanted
Then an "ArgumentError" should be thrown

Problem

There are two issues I have to address:

1. The first step should not fail when an exception is thrown.
2. The exception that the first step throws should be accessible to the second step in order to do some assertion magic.

Unelegant And Cumbersome Solution

The best approach I've been able to come up with is caching the exception in the first step and putting it into an instance variable that the second step can access, like so:

When /^I do something unwanted$/ do
  begin
    throw_an_exception!
  rescue => @error
  end
end

Then /^an "(.*)" should be thrown$/ do |error|
  @error.class.to_s.should == error
end

However, this makes the first step more or less useless in cases where I don't want it to fail, and it requires an instance variable, which is never a good thing.

So, can anyone help me out with an at least less cumbersome solution? Or should I write my features differently anyway? Any help would be much appreciated.

Answer 1

I thought about it once more, and maybe the answer is:

There is no elegant solution, because the Given-When-Then-Scheme is violated in your case. You expect that "Then an exception should be thrown" is the outcome of "When I do something unwanted".

But when you think about it, this is not true! The exception is not the outcome of this action, in fact the exception just shows that the "When"-Statement failed.

My solution to this would be to test at a higher level:

When I do something unwanted
Then an error should be logged

or

When I do something unwanted
Then the user should get an error message

or

When I do something unwanted
Then the program should be locked in state "error"

or a combination of these.

Then you would "cache the exception" in your program - which makes perfect sense, as you most likely need to do that anyway.

The two problems you've stated would be solved, too.

In case you really must test for exceptions

Well, i guess then cucumber isn't the right test suite, hmm? ;-)

As the Given-When-Then-Scheme is violated anyway, I would simply write

When I do something unwanted it should fail with "ArgumentError"

and in the step definitions something like (untested, please correct me if you try it)

When /^I do something unwanted it should fail with "(.*)"$/ do |errorstring|
  expect {
    throw_an_exception!
  }.to raise_error(errorstring)
end

As said above, that is horribly wrong as the scheme is broken, but it would serve the purpose, wouldn't it? ;-)

You'll find further documentation at testing errors at rspec expectations.

Answer 2

One option is to mark the scenario with @allow-rescue and check the page's output and status code. For example

In my_steps.rb

Then(/^the page (?:should have|has) content (.+)$/) do |content|
  expect(page).to have_content(content)
end

Then(/^the page should have status code (\d+)$/) do |status_code|
  expect(page.status_code.to_s).to eq(status_code)
end

Then /^I should see an error$/ do
  expect(400..599).to include(page.status_code)
end

In my_feature.feature

@allow-rescue
Scenario: Make sure user can't do XYZ
  Given some prerequisite
  When I do something unwanted
  Then the page should have content Routing Error
  And the page should have status code 404

or alternatively:

@allow-rescue
Scenario: Make sure user can't do XYZ
  Given some prerequisite
  When I do something unwanted
  Then I should see an error

This may not be exactly what you were hoping for, but it might be an acceptable workaround for some people who come across this page. I think it will depend on the type of exception, since if the exception is not rescued at any level then the scenario will still fail. I have used this approach mostly for routing errors so far, which has worked fine.

Answer 3

It is possible to raise an exception in a When block and then make assertions about it in the following Then blocks.

Using your example:

When /^I do something unwanted$/ do
  @result = -> { throw_an_exception! }
end

Then /^an "(.*)" should be thrown$/ do |error|
  expect{ @result.call }.to raise_error(error)
end

That example uses RSpec's matchers but the important part is the -> (Lambda); which allows the reference to the throw_an_exception! method to be passed around.

I hope that helps!

Answer 4

I'm answering from the perspective of someone who uses Cucumber features in a Behavior-Driven Development situation, so take it or leave it...

Scenarios should be written to test a 'feature' or functionality of the application, as opposed to being used to test the code itself. An example being:

When the service is invoked
Then a success code should be returned

It sounds like your test case (i.e. If I do this, then this exception should be thrown) is a candidate for unit or integration testing - in my case, we would use some Mocking or unit testing framework.

My suggestion would be to re-evaluate your feature scenarios to see if they are really testing what you intend them to test. From personal experience, I've found that if my test classes are becoming abnormally complex, then my features are 'wrong.'