Rspec false positive because failure exception is rescued in code being tested

Question

I have an rspec test that I expect to fail, but it is passing because the code that it is testing rescues the exception that rspec raises. Here's an example of the situation:

class Thing do

  def self.method_being_tested( object )
    # ... do some stuff

    begin
      object.save!
    rescue Exception => e
      # Swallow the exception and log it
    end
  end

end

In the rspec file:

describe "method_being_tested" do
  it "should not call 'save!' on the object passed in" do
    # ... set up the test conditions

    mock_object.should_not_receive( :save! )
    Thing.method_being_tested( mock_object )
  end
end

I knew that the execution was reaching the "object.save!" line of the method being tested, and the test should therefore be failing, but the test passes. Using the debugger in the rescue block, I find the following:

(rdb:1) p e # print the exception object "e"
#<RSpec::Mocks::MockExpectationError: (Mock "TestObject_1001").save!
    expected: 0 times
    received: 1 time>

So basically the test is failing but, but the failure is being suppressed by the very code it is trying to test. I cannot figure out a viable way to stop this code from swallowing Rspec exceptions without somehow compromising the code. I don't want the code to explicitly check if the exception is an Rspec exception, because that is bad design (tests should be written for code, code should never be written for tests). But I also can't check that the exception is any particular type that I DO want it to catch, because I want it to catch ANYTHING that could be raised in a normal production environment.

Someone must have had this problem before me! Please help me find a solution.

You're asking if there's a way to catch all exceptions, except the one kind you don't want to catch, without checking what type of exception it is in the catching code. The answer here is no. — Brett Bender, Sep 15 '11 at 16:36

jimworm · Answer 1 · 2013-12-20T00:02:29.470

Assuming the code is correct as-is:

describe "method_being_tested" do
  it "should not call 'save!' on the object passed in" do
    # ... set up the test conditions
    calls = 0
    mock_object.stub(:save!) { calls += 1 }
    expect {Thing.method_being_tested(mock_object)}.to_not change{calls}
  end
end

If there's no need to catch absolutely all exceptions including SystemExit, NoMemoryError, SignalException etc (input from @vito-botta):

begin
  object.save!
rescue StandardError => e
  # Swallow "normal" exceptions and log it
end

StandardError is the default exception level caught by rescue.

score 1 · Answer 2 · answered Jun 20 '13 at 07:08

1

I would refactor it like so:

class Thing do

  def self.method_being_tested!( object )

    # ... do some stuff

    return object.save
  end

end

If you want to ignore the exception thrown by save! there is no point in calling save! in the first place. You just call save and inform the calling code accordingly.

answered Jun 20 '13 at 07:08

Lefteris Laskaridis

2,292
2
24
38

I'm not sure if this is a general solution, if the save! is instead an API call there will still be the problem of handling the exceptions – xxjjnn Dec 19 '13 at 10:51

score 1 · Answer 3 · answered Sep 16 '11 at 02:45

1

from rspec-mock:

module RSpec
  module Mocks
    class MockExpectationError < Exception
    end

    class AmbiguousReturnError < StandardError
    end
  end
end

Do you really need to catch Exception? Could you catch StandardError instead?

Catching all exceptions is generally a bad thing.

answered Sep 16 '11 at 02:45

zetetic

47,184
10
111
119

@zetetic Arguably, catching and ignoring Exceptions is a bad thing. It could be perfectly legal to catch any (or all) exceptions AND act upon them. – Lefteris Laskaridis Jun 20 '13 at 07:00

Rspec false positive because failure exception is rescued in code being tested

3 Answers3