python testing using hypothesis

Question

The package hypothesis provides a rich set of strategies to use, if one wants to test against known type of input arguments.

Consider the following class and one of its test using hypothesis:

from hypothesis.strategies import floats, integers, one_of
from hypothesis import given
import unittest

class A:
    def __init__(self, value=0):
        assert isinstance(value, (int, float))
        self.value = value

    @property
    def doubled(self):
        return self.value * 2


class Test_A(unittest.TestCase):

    @given(testval=one_of(floats(allow_infinity=False, allow_nan=False), integers()))
    def test_A_instantiation(self, testval):
        self.assertEqual(testval * 2, A(value=testval).doubled)


if __name__ == '__main__':
    unittest.main()

This checks two things: implicitly, if the provided value is of the accepted types and explicitly, if the calculation in self.double is correct.

This test class can be considered incomplete at least because any input type other than float or int is not checked: for that one needed another test case, where the assertion error is checked against directly, let's say using AssertRaises.

My question is: how do you do that in the hypothesis testing phiolosophy?

Hypothesis doesn't seem to have the a "not_one_of" strategy, that is, one where you can define something like "all builtin types except for this-and-that types". This leaves only the possibility to define a "one_of" strategy with a rally long list of types to choose from.

Note: this whole situation reminds me of the root problem in Jurassic Park, when they checked if the have the expected number of dinos, but never checked if they have more. So I don't think there is no simple solution, but I first tried hypothesis yesterday so it may be my lack of knowledge.

Such a test can never be complete. I would consider one single test with any other input as sufficient. Especially since you are testing àssert isinstance` — Sosel, Oct 25 '17 at 06:31
Fair point: isinstance basically rules out all other types and covers all cases... if types are correctly defined, which should also be tested, right? But I think I get the point: that would be like testing the builtin assert. So we can say that the test case above is more or less correct and enough? — jake77, Oct 25 '17 at 06:36
I would add one test, where you test e.g. using a string instead of int and float, so one single test case where assert should fail. — Sosel, Oct 25 '17 at 07:01
This is how I organized my tests before first meeting hypothesis. I guess I'll just have to adjust mentally to the possibilites of hypothesis and where to use them. Would you care to write an answer to become the accepted or shall I do it? — jake77, Oct 25 '17 at 07:10

score 1 · Accepted Answer · answered Oct 25 '17 at 07:22

1

Testing above case can never be complete, especially since the main point would be about testing an assertion. Therefore as discussed above a single test where assertion should fail should be sufficient.

answered Oct 25 '17 at 07:22

Sosel

1,678
1
16
31

score 0 · Answer 2 · answered Mar 26 '19 at 13:07

0

An alternative would be to use the filter method provided by hypothesis. Then you can generate Any object and filter out those cases that you don't want. Since you will only filter a small portion, this will still be performant.

answered Mar 26 '19 at 13:07

Ivo Merchiers

1,589
13
29

python testing using hypothesis

2 Answers2