How to combine the @fixture and the @given decorators in Python hypothesis testing?

Question

I'm working with pytest and hypothesis for property-based testing in Python.

As a minimal example, suppose I want to test that my Matrix class constructor correctly assign the number of rows and columns to a Matrix instance. I want to do it in two separate test functions but I also want them to share the same object to avoid recreating it twice (this can be done by using the pytest.fixture decorator). However, at the same time, I want to use the hypothesis.given decorator to make my test parameterized by an integer seed randomly generated by hypothesis.strategies.integers.

Is it possible to combine these two features? If not, is there any workaround to achieve this?

The code below is a minimal working example with my test_nrows and test_ncols functions sharing the same object created by the matrix fixture.

import numpy as np
from pytest import fixture

class Matrix:
    def __init__(self, nrows, ncols, seed=None):
        np.random.seed(seed)
        self.array = np.random.rand(nrows, ncols)

@fixture(scope='module')
def matrix():
    return Matrix(nrows=2, ncols=3)

def test_nrows(matrix):
    assert matrix.array.shape[0] == 2

def test_ncols(matrix):
    assert matrix.array.shape[1] == 3

I want to make the tests parameterized rewriting the fixture function as something like the following:

from hypothesis import given, strategies

@given(seed=strategies.integers())
@fixture(scope='module')
def matrix(seed):
    return Matrix(nrows=2, ncols=3, seed=seed)

However, this fails with the error message fixture 'matrix' not found.

Nope, it doesn't but it gives a different error: `Can't apply @pytest.fixture() to matrix because it is already decorated with @hypothesis.given()` — SimoneGasperini, Mar 16 '23 at 16:42

score 1 · Answer 1 · edited Mar 22 '23 at 18:48

You're not supposed to combine @hypothesis.given and @pytest.fixture like that. The way to implement random matrix draws is by defining your own composite strategy, not a fixture.

Let's say this was your Matrix class, as in your example,

import typing as tp

import numpy as np
import numpy.typing as npt
    

class Matrix:

    array: npt.NDArray[np.float_]

    def __init__(self, nrows: int, ncols: int, seed: int) -> None:
        np.random.seed(seed)
        self.array = np.random.rand(nrows, ncols)

then your composite hypothesis search strategy would be defined as follows:

from __future__ import annotations

import typing as tp

import hypothesis.strategies as st
import numpy as np


if tp.TYPE_CHECKING:
    import numpy.typing as npt
    from hypothesis.strategies import SearchStrategy


seed_strategy: tp.Final[SearchStrategy[int]] = st.integers(
    # Bounds given are for the acceptable range of `numpy.random.seed`
    min_value=0, max_value=2**32 - 1
)


@st.composite
def drawMatrix(
    drawFromStrategy: st.DrawFn, /, *, nrows: int = 2, ncols: int = 3
) -> Matrix:

    """
    Strategy which draws random `numpy.float_` matrices of dimensions (nrows × ncols)

    Parameters
    ----------
    drawFromStrategy
        Callable which draws items from other hypothesis search strategies
    nrows
        Number of rows for the matrix
    ncols
        Number of columns for the matrix

    Returns
    -------
    Matrix
        Matrix for testing
    """

    seed: int = drawFromStrategy(seed_strategy)
    return Matrix(nrows=nrows, ncols=ncols, seed=seed)

Your pytest tests would then be written as the following:

_2x3_matrix_strategy: tp.Final[SearchStrategy[Matrix]] = drawMatrix()


@hypothesis.given(matrix=_2x3_matrix_strategy)
def test_nrows(matrix: Matrix) -> None:
    assert matrix.array.shape[0] == 2


@hypothesis.given(matrix=_2x3_matrix_strategy)
def test_ncols(matrix: Matrix) -> None:
    assert matrix.array.shape[1] == 3


@hypothesis.given(matrix=drawMatrix(nrows=3, ncols=4))
def test_bad_dimensions(matrix: Matrix) -> None:
    assert matrix.array.shape[0] == 2
    assert matrix.array.shape[1] == 3

Demonstration of this in action:

platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0
rootdir: ..., configfile: pytest.ini
plugins: hypothesis-6.54.5, anyio-3.6.2
collected 3 items                                                                                                                                                                                                                                                       

test_matrix.py ..F                                                                                                                                                                                                                                  [100%]

=============================================================================================================================== FAILURES ================================================================================================================================
__________________________________________________________________________________________________________________________ test_bad_dimensions __________________________________________________________________________________________________________________________

    @hypothesis.given(matrix=drawMatrix(nrows=3, ncols=4))
>   def test_bad_dimensions(matrix: Matrix) -> None:

test_matrix.py:91: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

matrix = <test_matrix.Matrix object at 0x7f1dfb263d90>

    @hypothesis.given(matrix=drawMatrix(nrows=3, ncols=4))
    def test_bad_dimensions(matrix: Matrix) -> None:
>       assert matrix.array.shape[0] == 2
E       assert 3 == 2
E       Falsifying example: test_bad_dimensions(
E           matrix=<test_matrix.Matrix at 0x7f1dfb263d90>,
E       )

test_matrix.py:92: AssertionError
======================================================================================================================== short test summary info ========================================================================================================================
FAILED test_matrix.py::test_bad_dimensions - assert 3 == 2
====================================================================================================================== 1 failed, 2 passed in 0.46s ======================================================================================================================

Please see the source code for the signature expected of the function decorated by @st.composite. In short, the first parameter must take a positional argument which indicates the hypothesis drawing function. This parameter is "swallowed" when you decorate this with @st.composite to make strategy factory callables (kind of like how self is "swallowed" when calling instance methods).

How to combine the @fixture and the @given decorators in Python hypothesis testing?

1 Answers1