I am writing an application that performs REST operations using Kenneth Reitz's requests library and I'm struggling to find a nice way to unit test these applications, because requests provides its methods via module-level methods.
What I want is the ability to synthesize the conversation between the two sides; provide a series of request assertions and responses.
It is in fact a little strange that the library has a blank page about end-user unit testing, while targeting user-friendliness and ease of use. There's however an easy-to-use library by Dropbox, unsurprisingly called responses. Here is its intro post. It says they've failed to employ httpretty, while stating no reason of the fail, and written a library with similar API.
import unittest
import requests
import responses
class TestCase(unittest.TestCase):
@responses.activate
def testExample(self):
responses.add(**{
'method' : responses.GET,
'url' : 'http://example.com/api/123',
'body' : '{"error": "reason"}',
'status' : 404,
'content_type' : 'application/json',
'adding_headers' : {'X-Foo': 'Bar'}
})
response = requests.get('http://example.com/api/123')
self.assertEqual({'error': 'reason'}, response.json())
self.assertEqual(404, response.status_code)
If you use specifically requests try httmock. It's wonderfully simple and elegant:
from httmock import urlmatch, HTTMock
import requests
# define matcher:
@urlmatch(netloc=r'(.*\.)?google\.com$')
def google_mock(url, request):
return 'Feeling lucky, punk?'
# open context to patch
with HTTMock(google_mock):
# call requests
r = requests.get('http://google.com/')
print r.content # 'Feeling lucky, punk?'
If you want something more generic (e.g. to mock any library making http calls) go for httpretty.
Almost as elegant:
import requests
import httpretty
@httpretty.activate
def test_one():
# define your patch:
httpretty.register_uri(httpretty.GET, "http://yipit.com/",
body="Find the best daily deals")
# use!
response = requests.get('http://yipit.com')
assert response.text == "Find the best daily deals"
HTTPretty is far more feature-rich - it offers also mocking status code, streaming responses, rotating responses, dynamic responses (with a callback).
You could use a mocking library such as Mocker to intercept the calls to the requests library and return specified results.
As a very simple example, consider this class which uses the requests library:
class MyReq(object):
def doSomething(self):
r = requests.get('https://api.github.com', auth=('user', 'pass'))
return r.headers['content-type']
Here's a unit test that intercepts the call to requests.get and returns a specified result for testing:
import unittest
import requests
import myreq
from mocker import Mocker, MockerTestCase
class MyReqTests(MockerTestCase):
def testSomething(self):
# Create a mock result for the requests.get call
result = self.mocker.mock()
result.headers
self.mocker.result({'content-type': 'mytest/pass'})
# Use mocker to intercept the call to requests.get
myget = self.mocker.replace("requests.get")
myget('https://api.github.com', auth=('user', 'pass'))
self.mocker.result(result)
self.mocker.replay()
# Now execute my code
r = myreq.MyReq()
v = r.doSomething()
# and verify the results
self.assertEqual(v, 'mytest/pass')
self.mocker.verify()
if __name__ == '__main__':
unittest.main()
When I run this unit test I get the following result:
.
----------------------------------------------------------------------
Ran 1 test in 0.004s
OK
Missing from these answers is requests-mock.
From their page:
>>> import requests >>> import requests_mockAs a context manager:
>>> with requests_mock.mock() as m: ... m.get('http://test.com', text='data') ... requests.get('http://test.com').text ... 'data'Or as a decorator:
>>> @requests_mock.mock() ... def test_func(m): ... m.get('http://test.com', text='data') ... return requests.get('http://test.com').text ... >>> test_func() 'data'
using mocker like in srgerg's answer:
def replacer(method, endpoint, json_string):
from mocker import Mocker, ANY, CONTAINS
mocker = Mocker()
result = mocker.mock()
result.json()
mocker.count(1, None)
mocker.result(json_string)
replacement = mocker.replace("requests." + method)
replacement(CONTAINS(endpoint), params=ANY)
self.mocker.result(result)
self.mocker.replay()
For the requests library, this would intercept the request by method and endpoint you're hitting and replace the .json() on the response with the json_string passed in.
If you break out your response handler/parser into a separate function, you can work with requests.Response objects directly, without needing to mock the client-server interaction.
Code under test
from xml.dom import minidom
from requests.models import Response
def function_under_test(s3_response: Response):
doc = minidom.parseString(s3_response.text)
return (
s3_response.status_code,
doc.getElementsByTagName('Code').item(0).firstChild.data,
)
Test code
import unittest
from io import BytesIO
class Test(unittest.TestCase):
def test_it(self):
s3_response = Response()
s3_response.status_code = 404
s3_response.raw = BytesIO(b"""<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>NoSuchKey</Code>
<Message>The resource you requested does not exist</Message>
<Resource>/mybucket/myfoto.jpg</Resource>
<RequestId>4442587FB7D0A2F9</RequestId>
</Error>
""")
parsed_response = function_under_test(s3_response)
self.assertEqual(404, parsed_response[0])
self.assertEqual("NoSuchKey", parsed_response[1])
There's a library for this, if you want to write your test server with Flask: requests-flask-adaptor
You just have to be careful with the order of imports when monkeypatching.
