Python hypothesis: How to assert errors in test function

Question

I'm just beginning to use hypothesis for test generation. How can I allow certain errors in the test function when I use hypothesis?

Simple example is a division function which works for integers > 0:

def divide(a: int, b: int) -> float:
    return a/b

When I write a hypothesis test without explicitly excluding 0 I get errors of course:

from hypothesis import given, strategies as st

@given(a=st.integers(), b=st.integers())
def test_divide(a, b):
    assert isinstance(divide(a,b), float)

Since hypothesis uses the integer 0 I get ZeroDivisionError:

---------------------------
Falsifying example: test_divide(
    a=0, b=0,
)
============================
FAILED test.py::test_divide - ZeroDivisionError: division by zero

This is to be expected of course and in this simple example I could simply exclude 0 because I know the error will happen. In other functions, especially the ones working with strings, it would be difficult to exclude all things where I know that an error is thrown. In many cases though the error is ok, e.g. when using null bytes in string functions.

Is it possible to somehow 'allow' certain types of errors?

Or do I have to rethink how to write tests when using hypothesis?

just exclude zero from your strategy for divisors, like `b=st.integers().filter(bool)` — Azat Ibrakov, Jun 23 '21 at 14:47
and test zero separately in a different test case with `b=st.just(0)` — Azat Ibrakov, Jun 23 '21 at 14:48

score 0 · Answer 1 · answered Jun 16 '21 at 11:04

0

If I understand correctly, this is what you're looking for

def divide(a: int, b: int) -> float:
    try:
        return a/b
    except ZeroDivisionError:
        return # the value you want to return for this scenario

answered Jun 16 '21 at 11:04

I Dav

374
1
2
8

I don't want to return anything, I think the `ZeroDivisionError` is ok here. I would like to assert a value or a specific error in the test function. The example is somewhat constructed because you can test without 0 and write a second test that asserts the proper error. More difficult though for other types/functions. – Martin Preusse Jun 16 '21 at 11:21
Instead of `return`, you could use `pass`. Even better, use [`with contextlib.suppress(ZeroDivisionError):`](https://docs.python.org/3/library/contextlib.html#contextlib.suppress) – Zac Hatfield-Dodds Jun 20 '21 at 14:25

score 0 · Answer 2 · answered Aug 09 '21 at 19:08

Not an expert here, but from my tinkering with hypothesis I think the preferred way to skip this known class of errors would be use of assume.

from hypothesis import assume, given, strategies as st

@given(a=st.integers(), b=st.integers())
def test_divide(a, b):
    assume(b != 0)
    assert isinstance(divide(a,b), float)

Python hypothesis: How to assert errors in test function

2 Answers2