2

Using side_effect, I am trying to raise an exception when a mock is called but I get a DID NOT RAISE EXCEPTION error that I do not understand.

Based largely on this answer, I have created a simple example where there is a Query class with a class method make_request_and_get_response which can raise several exceptions. These exceptions are being handled within the get_response_from_external_api method in main.py.

query.py

from urllib.request import urlopen
import contextlib
import urllib

class Query:
    def __init__(self, a, b):
        self.a = a
        self.b = b

        self.query = self.make_query()


    def make_query(self):
        # create query request using self.a and self.b
        return query


    def make_request_and_get_response(self):  # <--- the 'dangerous' method that can raise exceptions
        with contextlib.closing(urlopen(self.query)) as response:
            return response.read().decode('utf-8')

main.py

from foo.query import *

def get_response_from_external_api(query):
    try:
        response = query.make_request_and_get_response()
    except urllib.error.URLError as e:
        print('Got a URLError: ', e)
    except urllib.error.HTTPError as e:
        print('Got a HTTPError: ', e)
    # {{various other exceptions}}
    except Exception:
        print('Got a generic Exception!')
        # handle this exception


if __name__ == "__main__":    
    query = Query('input A', 'input B')
    result = get_response_from_external_api(query)
    return result

Using pytest, I am trying to mock that 'dangerous' method (make_request_and_get_response) with a side effect for a specific exception. Then, I proceed with creating a mocked Query object to use when calling the make_request_and_get_response with the expectation that this last call give a 'URLError' exception.

test_main.py

import pytest
from unittest.mock import patch
from foo.query import Query
from foo.main import get_response_from_external_api


class TestExternalApiCall:
    @patch('foo.query.Query')
    def test_url_error(self, mockedQuery):
        with patch('foo.query.Query.make_request_and_get_response', side_effect=Exception('URLError')):
            with pytest.raises(Exception) as excinfo:
                q= mockedQuery()
                foo.main.get_response_from_external_api(q)
            assert excinfo.value = 'URLError'
            # assert excinfo.value.message == 'URLError' # this gives object has no attribute 'message'

The test above gives the following error:

>       foo.main.get_response_from_external_api(q)
E       Failed: DID NOT RAISE <class 'Exception'> id='72517784'>") == 'URLError'
timmy78h
  • 183
  • 1
  • 3
  • 10

1 Answers1

1

Pytest cannot detect an exception because in get_response_from_external_api you are catching all of them. Depending on your requirements, you have these options:

  1. Don't catch the exceptions in get_response_from_external_api.
  2. Catch the exceptions, do whatever you want and then re-raise them.
  3. Instead of detecting an exception with pytest.raises, use capsys fixture to capture what is being printed and make assertions on that output.
tmt
  • 7,611
  • 4
  • 32
  • 46
  • I am very much confused. In the interest of having expectation for any external dependency as is this call to the external api, should I not be catching all of the exceptions in `get_response_from_external_api` as I currently do? That is, isn't this the 'correct' approach programmatically? Furthermore, is it only `pytest`'s shortcoming and is another lib (e.g. `unittest.mock` able to detect the exceptions? Apologies if my questions are silly but I am very keen on understanding what the best approach in handling this external api call and properly/fully testing it using mocks. – timmy78h Nov 13 '19 at 18:55
  • Also, just for the benefit of an example, could you please show me how one could mock and raise an exception if we assume that the `get_response_from_external_api` does not catch all exceptions? – timmy78h Nov 13 '19 at 19:14
  • @timmy78h You are setting the side effect correctly as far as I can see. Look at it this way: you are calling `get_response_from_external_api()` and you expect that the exception would rise from it even though it originates in `make_request_and_get_response()` for which you are setting the side effect. But it doesn't rise out of that `get_response_from_external_api()` because the exception is being caught in it. This is not an issue with pytest, the Python's unittest would give you the same. – tmt Nov 13 '19 at 19:21
  • As for whether you are supposed to be catching the exceptions: that's a design decision that I cannot answer for you because I don't know what handling you do there. From your example I see only a call to another method and print statements so it's hard to say what's actually testable on that function. – tmt Nov 13 '19 at 19:22
  • Thank you, I understand now. But then, can you suggest/advise on how one can test the `get_response_from_external_api` exception cases? I thought that mocking the `make_request_and_get_response` should suffice but I was wrong. How can one test the exceptions of the `make_request_and_get_response`? – timmy78h Nov 13 '19 at 19:26
  • The print statements are there to basically echo back the error given when the attempt to `make_request_and_get_response()`. There is not much more than this. The design decision I made was to handle every error and pass information back to the caller and use a mock for the 'dangerous' method to allow me test my design. I am new to this so I cannot see how else I could have done that: if you have any design suggestions for me to consider that would very very helpful. – timmy78h Nov 13 '19 at 19:30
  • @timmy78h You are testing `get_response_from_external_api()`. From what I can see it calls another method and makes `print` statements so these are pretty much the only things you can test for. Assert that `make_request_and_get_response()` was called (the mock object has a property which tracks if an how many times the it was called) and assert what is being printed with my option #3. As to what would be a good design for your application: that's something beyond the scope of SO questions/answers. I would be like a doctor giving diagnosis over Internet without seeing the patient. – tmt Nov 13 '19 at 19:35
  • I understand - you've been most helpful albeit I am still have to learn a lot. I will try to find a way to use the print statements to test even though that to me sounds like poor testing/design. Thanks again tmt! – timmy78h Nov 13 '19 at 20:03
  • Just a follow-up and for completeness purposes, if you could edit your answer to include an example for asserting that the method was called + asserting what is printed (#3), that would be very educational for me. – timmy78h Nov 13 '19 at 20:24