Parametrized tests in pytest with varying marks for different test functions

Question

I am currently trying to use pytest's parameterization feature in the following context:

I have multiple functions that should be tested with a universal set of test cases. Depending on the tested function, the same test case should either pass or xfail. I came up with a silly example to illustrate this:

import pytest


# Functions to test
def sum_int(a, b):
    assert isinstance(a, int)
    assert isinstance(b, int)
    return a + b


def sum_any(a, b):
    return a + b


# Universal test cases
TESTCASES = [
    "a,b,result", [
        (1, 1, 2),
        ("a", "a", "aa")
        ]
    ]


class Tests:
    @pytest.mark.parametrize(*TESTCASES, ids=["int_pass", "str_fail"])
    def test_sum_int(self, a, b, result):
        assert sum_int(a, b) == result

    @pytest.mark.parametrize(*TESTCASES, ids=["int_pass", "str_pass"])
    def test_sum_any(self, a, b, result):
        assert sum_any(a, b) == result

Unfortunately, it seems not to be possible to just pass additional marks (like pytest.mark.xfail(reason=AssertionError) to parametrize() like it can be done with IDs.

# Does not work
@pytest.mark.parametrize(*TESTCASES,
                         ids=["int_pass", "str_fail"],
                         marks=[None, pytest.mark.xfail(reason=AssertionError)])
def test_sum_int(self, a, b, result):
    assert sum_int(a, b) == result

What could be a good way to achieve this?

score 2 · Accepted Answer · answered Feb 16 '21 at 18:30

I had to discover that the answer to my problem is relatively simple. The pytest.param mechanism allows to specify marks for specific test cases:

@pytest.mark.parametrize(
    TESTCASES[0],
    [
        pytest.param(*args, marks=marks)
        for args, marks
        in zip(TESTCASES[1], [[], [pytest.mark.xfail(reason=AssertionError)]])
    ],
    ids=["int_pass", "str_fail"],
    )
def test_sum_int(self, a, b, result):
    assert sum_int(a, b) == result

score 0 · Answer 2 · answered Aug 27 '20 at 10:09

0

If you are trying to mark only the sum of string test you can do it like this:

# Universal test cases
TESTCASES = [
    "a,b,result", [
        (1, 1, 2),
        pytest.mark.xfail(("a", "a", "aa"))
        ]
    ]

Have a read here https://docs.pytest.org/en/2.8.7/parametrize.html#:~:text=#%20content%20of%20test_expectation.py

answered Aug 27 '20 at 10:09

lucrib

448
1
9
21

Thanks for your answer but this does not cut it. I want to avoid defining test cases for the functions separately. Doing it like this will give an unexpected success for `sum_any`. To make it a bit more clear, I would also like to use maybe the `pytest.mark.raises()` marker from the pytest-raises plugin instead, which would make `sum_any` fail then. – Jan Joswig Aug 27 '20 at 10:55

Parametrized tests in pytest with varying marks for different test functions

2 Answers2