0

I am creating my own fixture to simulate a service endpoint needed in my unit tests. In order to intercept the HTTP requests, I use requests_mock as follows:

@pytest.fixture
def sparql_endpoint(requests_mock):
    yield lambda uri, initial_data: Endpoint(requests_mock, uri, initial_data)

and in Endpoint.__init__ I do the following:

    m.post(url=uri, raw=self.handle_post)
    m.get(url=uri, raw=self.handle_get)

In my actual testcase I inject the endpoint and initialize it:

def test_basic_select(my_endpoint):
    repo_uri = 'https://my.rdfdb.com/repo/sparql'
    rdf_files = ['tests/upper_ontology.ttl',
                 'tests/domain_ontology.ttl',
                 'tests/instance_data.ttl']
    endpoint = sparql_endpoint(repo_uri, rdf_files)

Which does, in fact, initialize the mocked endpoint and I see Mocker.start() get invoked if I set a breakpoint there. However, later in the testcase I get the following:

..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:222: in urlopen
    return opener.open(url, data, timeout)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:525: in open
    response = self._open(req, data)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:543: in _open
    '_open', req)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:503: in _call_chain
    result = func(*args)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:1360: in https_open
    context=self._context, check_hostname=self._check_hostname)

E               urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>

..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:1319: URLError

Because it cannot resolve the fake URL I gave it. So, did I somehow mess up the handler registration so that the Matcher is not kicking the request there? Why is urlopen still trying to resolve the host?

user2650994
  • 136
  • 5

2 Answers2

0

It would appear you're not starting the mocker anywhere? I don't have a lot of experience with pytest, but the fact that the request is getting through to urllib implies it never hit the mock object.

Have you got a self contained example that can be run? I'd guess it's to do with the way you are yielding the requests_mock object. The mocker will works like a normal context manager and so will stop when you exit the scope, but it's hard to say without tracing it.

jamielennox
  • 368
  • 1
  • 2
  • 9
  • The mock was being injected via the `requests_mock` pytest fixture, which was being injected into the method. I have since figured out my issue, will add an answer. – user2650994 Oct 06 '21 at 11:58
0

I have figured out the issue, which was due to the underlying code (SPARQLWrapper) not using requests, instead using urlopen directly, and thus bypassing my mock. In order to be able to intercept both types of access, I resorted to using HTTPretty, which did a more thorough mocking. The code ended up looking as follows:

First, in the fixture itself, I enabled the mocking:

    httpretty.set_default_thread_timeout(60)
    # enable HTTPretty so that it will monkey patch the socket module
    httpretty.enable(verbose=True, allow_net_connect=False)

    yield lambda uri, initial_data, **kwargs: Endpoint(uri, initial_data, **kwargs)

    # disable afterwards, so that you will have no problems in code that uses that socket module
    httpretty.disable()
    # reset HTTPretty state (clean up registered urls and request history)
    httpretty.reset()

Note that I yield a lambda that creates the instance implementing the fixture, so after it is cleaned up, httpretty can clean up as well.

In the fixture initialization, I create the bindings:

        httpretty.register_uri(httpretty.GET, uri,
                               body=self.handle_get)
        httpretty.register_uri(httpretty.POST, uri,
                               body=self.handle_post)

When I inject the fixture, I had to make an extra call to actually create it:

def test_request_get(sparql_endpoint):
    repo_uri = 'https://my.rdfdb.com/repo/sparql'
    rdf_files = ['tests/upper_ontology.ttl',
                 'tests/domain_ontology.ttl',
                 'tests/instance_data.ttl']
    # This calls the lambda defined in the fixture
    endpoint = sparql_endpoint(repo_uri, rdf_files)
user2650994
  • 136
  • 5