How does pytest create fixtures, or, how to get my yield?

Question

I have a generator function

def foo():
    resource = setup()
    yield resource
    tidy(resource)

I'm using this as a fixture

@pytest.fixture
def foofix():
    yield from foo()

This works fine.

I want to test foo

def test_foo():
    res_gen = foo()
    res = next(res_gen)
    assert res.is_working

but because I tidy up resource immediately after I yield it, it's not longer available to assert it's working. How then does pytest use resource before it's tidied up, and how can I do the same?

Did you try having `test_foo` *actually use* the `foofix` fixture? — Karl Knechtel, Aug 21 '23 at 18:24
@KarlKnechtel i'm just trying that now, but seeing likely unrelated errors — joel, Aug 21 '23 at 18:24
@KarlKnechtel yeah that works, but I'm still curious about how pytest does it — joel, Aug 21 '23 at 18:43
... I'm a little confused. If you were expecting this to work *without* having `test_foo` use the fixture, then *what was the intended point* of creating the fixture? Oh, you mean that the question is about *how to do the same thing that Pytest does here*, and is *not actually about* testing, but is purely about understanding generators? (I would say to look at the source on GitHub, but it appears to be quite complex.) — Karl Knechtel, Aug 21 '23 at 19:07
@KarlKnechtel yeah it's about understanding generators. My actual use case is I'm reusing `foo` across multiple projects, but want to export it as a function not a fixture so I can customize it without using pytest, so I tried to test it on its own — joel, Aug 21 '23 at 20:54
test_foo should run fine. The next() does not call tidy(). Only a subsequent next() will. — Bharel, Aug 21 '23 at 23:55

score 0 · Answer 1 · answered Aug 31 '23 at 21:47

If you want to test foo, you have just to call it in your code, not to wrap it in a pytest fixture.

Fixtures are good for values that require some boiler plate to be generated, and, when these values have to be tidyed up after the test, in a transparent way to the test - so that the test can consume the value, and do not worry about any boiler plate.

If you want to test the foo generator itself, and not just consume its value in a test, there is no sense in wrapping it in a fixture - simply import it, or its module, in the module containing the test function, start the generator by calling it, and then call next and catch StopIteration if needed.

When you run

def test_foo():
    res_gen = foo()
    res = next(res_gen)
    assert res.is_working

as a plain Python generator, after calling next on foo() you will get the resource before it is disposed: execution in the generator is paused at the yield resource expression when assert res.is_working is executed. Only upon the subsequent call to next would the tidy(resource) line run.

Actually this is another problem in your design: generators do not, in any way, ensure they will run to completion (and therefore finishing your resource) - in a pattern like this, it is up to the caller to call next until the generator is exhausted.

It is different when you mark a generator as a pytest fixture - in that case, pytest will run the generator again, after the first yield, after the test is gone - but that is a pytest "goodie", since it uses this pattern to start and to clean-up a fixture, not the natural behavior of generators.

The pattern there is close to it is to decorate such a generator with Python's contextlib.contextmanager call, and then your generator is transformed into a context manager that can be used as an expression in the with command. When the with command is over, the generator code bellow the yield is executed.

So, maybe you want this:


from contextlib import contextmanager

@contextmanager
def foo():
    resource = setup()
    # try/finally block is needed, otherwise, if an exception
    # occurs where resource is used in a `with` block, 
    # the finalization code is not executed.
    try: 
        yield resource
    finally:
        tidy(resource)

def test_foo():
    with foo() as res:
        # here, you have "resource" and it has not terminated
        assert res.is_working
    # the end of the "with" block resumes the generator and frees the resource
    return  # even if it is an implicit return.

How does pytest create fixtures, or, how to get my yield?

1 Answers1