Is it possible to generate objects from type-annotated classes for testing?

Question

I had a look at hypothesis recently and used it like this:

import hypothesis.strategies as s
from hypothesis import given

@given(s.integers(min_value=-(10 ** 6), max_value=10 ** 6))
def test_factorize(an_integer):
    if an_integer == 0:
        # This is tested in `test_factorize_zero` and should throw an exception
        return
    factors = mpu.math.factorize(an_integer)
    product = 1
    for factor in factors:
        product *= factor
    assert product == an_integer

This is pretty cool. The main limitation I see is the strategy (e.g. s.integers), although there are a lot of strategies and I'm still learning which ones there are / how to use them properly.

Is there a strategy to generate objects, given a type-annotated class which uses pydantic?

My Try

from typing import Optional

from hypothesis import given
from hypothesis.strategies import from_type
from pydantic import BaseModel


class Adress(BaseModel):
    city: str
    street: str
    house_number: int
    postal_code: int


class Person(BaseModel):
    prename: str
    middlename: Optional[str]
    lastname: str
    address: Adress


@given(from_type(Person))
def test_me(person: Person):
    assert isinstance(person, Person)

when I save this as test_foo.py and execute pytest, I get:

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_me ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

    @given(from_type(Person))
>   def test_me(person: Person):

test_foo.py:20: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   pydantic.error_wrappers.ValidationError: 3 validation errors for Person
E   prename
E     field required (type=value_error.missing)
E   lastname
E     field required (type=value_error.missing)
E   address
E     field required (type=value_error.missing)

pydantic/main.py:283: ValidationError
---------------------------------------------------------------- Hypothesis ----------------------------------------------------------------
You can add @seed(42732672939050403878146949573829059697) to this test or run pytest with --hypothesis-seed=42732672939050403878146949573829059697 to reproduce this failure.

I would especially love if hypothesis generated a person without a middle name and a person with a middle name.

Answer 1

This won't work out of the box, because Hypothesis needs to know what types to use in order to construct a pydantic model, and we've no particular support for pydantic (I've literally only just heard of it from this post).

Roughly there are two things that could be done in order to support it:

1. pydantic could generate correctly typed __init__ methods. The signature of a pydantic __init__ method is __pydantic_self__, **data:Any, so there's no way for us to know what arguments it expects. If instead the type signature matched that of the field arguments, Hypothesis would work out of the box for this.
2. Someone could write a PR to add pydantic support to hypothesis's from_type implementation so that it knows how to construct pydantic models. We've already got custom support for attrs and dataclasses, this would mostly just look like that.

My preferred solution would be the first if pydantic developers are open to it.

In the meantime, you can use the builds function to define custom strategies for your models. You can then register it as the strategy for that type or just use it directly.