How to collect values from a parametrized test function in a single pytest-xdist run

Question

Using pytest-xdist I run a parametrized test function that has a variable I am interested in. I would like to store a value of the variable in a separate object and use that object later on to compose a test report.

I tried to use a session scope fixture that was supposed to run once right after all tests had finished and the resulting object had become available. At first glance, it did work fine, yet I found out that pytest-xdist does not have a builtin support for ensuring a session-scoped fixture is executed exactly once. This means that my storing object was most probably overwritten from scratch N times, where N is a number of pytest-xdist workers that shared my test. And this is not desirable behavior.

A short snippet to illustrate the question

# test_x.py

import pytest
from typing import Tuple
from typing import List

@pytest.mark.parametrize('x', list(range(4)))
def test_x(x):
    result = (str(x), x ** 2)  # <-- here I have a dummy result, say a tuple

def write_report(results: List[Tuple[str, int]]):
    # here I would like to have ALL the results that is `results` expected to be:
    # [('0', 0), ('1', 1), ('2', 4), ('3', 9)]
    # considering the fact test_x was paralleled by pytest-xdist to N > 1 workers.
    ...

I run it with pytest -n auto test_x.py

Is there another way to collect all result values in such sequentional multiprocessing test invocation? I will appreciate any help on the matter.

Edited

Yesterday I found a promising package pytest_harvest, but haven't made it work yet. Everything goes nice as long as no pytest-xdist involved. The wonderful results_bag fixture works as expected storing the value in need and returning it in the session results hook pytest_sessionfinish when it all stops. But when you add xdist workers suddenly no session results at all (it does return an empty dict though).

# conftest.py
from pytest_harvest import is_main_process
from pytest_harvest import get_session_results_dct

def pytest_sessionfinish(session):
    dct = get_session_results_dct(session)
    if is_main_process(session):
        print(dct) # prints an empty OrderedDict...

# test_parallel.py
import pytest

@pytest.mark.parametrize('x', list(range(3)))
def test_parallel(x, results_bag):
    results_bag.x = x

pytest -sv test_parallel.py -> OK

pytest -sv -n auto test_parallel.py -> an empty OrderedDict

Any thoughts how to make it work decently?

How to collect values from a parametrized test function in a single pytest-xdist run

0 Answers0