How can I use hypothesis, and pytest-tornado yield-based testing, in the same test case?

Question

I'm writing py.test tests of my code that uses the Tornado library. How can I use Hypothesis in my tests that involve coroutines and the IOLoop? I've been able to write yield-based tests without Hypothesis by using pytest-tornado's @pytest.mark.gen_test, but when I try to combine it with @given, I receive the following error:

FailedHealthCheck: Tests run under @given should return None, but test_both returned <generator object test_both at 0x7fc4464525f0> instead.

See http://hypothesis.readthedocs.org/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.return_value to the suppress_health_check settings for this test.

I'm pretty confident that this is a real problem and not just a question of disabling the health check, considering that the Hypothesis docs say

yield based tests simply won’t work.

Here's code that demonstrates my situation:

class MyHandler(RequestHandler):

    @gen.coroutine
    def get(self, x):
        yield gen.moment
        self.write(str(int(x) + 1))
        self.finish()


@pytest.fixture
def app():
    return Application([(r'/([0-9]+)', MyHandler)])


@given(x=strategies.integers(min_value=0))
def test_hypothesis(x):
    assert int(str(x)) == x


@pytest.mark.gen_test
def test_tornado(app, http_client, base_url):
    x = 123
    response = yield http_client.fetch('%s/%i' % (base_url, x))
    assert int(response.body) == x + 1


@pytest.mark.gen_test
@given(x=strategies.integers(min_value=0))
def test_both(x, app, http_client, base_url):
    response = yield http_client.fetch('%s/%i' % (base_url, x))
    assert int(response.body) == x + 1

test_hypothesis and test_tornado work fine, but I get the error with test_both because I'm using yield and Hypothesis together.

Changing the order of the decorators didn't change anything, probably because the gen_test decorator is simply an attribute mark.

Can I write tests of my Tornado-based code that use Hypothesis? How?

score 8 · Accepted Answer · answered May 15 '16 at 18:21

You can accomplish this by calling run_sync() on the io_loop py.test fixture of pytest-tornado. This can be used in place of yield:

@given(x=strategies.integers(min_value=0))
def test_solution(x, app, http_client, base_url, io_loop):
    response = io_loop.run_sync(
        lambda: http_client.fetch('%s/%i' % (base_url, x)))
    assert int(response.body) == x + 1

Or you can place the body of your test in a coroutine, so that it can continue to use yield, and call this coroutine with run_sync():

@given(x=strategies.integers(min_value=0))
def test_solution_general(x, app, http_client, base_url, io_loop):
    @gen.coroutine
    def test_gen():
        response = yield http_client.fetch('%s/%i' % (base_url, x))
        assert int(response.body) == x + 1
    io_loop.run_sync(test_gen)

How can I use hypothesis, and pytest-tornado yield-based testing, in the same test case?

1 Answers1