Pytest order test A, then test B and then test A again

Question

Is it possible to order tests the way, that test A will go first, then test B go second and then test A will go again as third? I am using pytest-order lib.

Tech stack: Python 3.8.5, pytest 6.1.0, pytest-order 1.0.1

Here is the code used:

import logging
import pytest


@pytest.mark.order(2)
def test_a():
    logging.info('test_a')
    pass


@pytest.mark.order(1)
@pytest.mark.order(3)
def test_b():
    logging.info('test_b')
    pass

But test B is not executed third time. Only once as first one due to the two order marks.

Output:

=================== test session starts ==================

collecting ... collected 2 items

test.py::test_a PASSED [ 50%]

test.py::test_b PASSED [100%]

=================== 2 passed in 0.07s ===================

pL3b · Accepted Answer · 2023-03-03T15:41:41.490

Actually pytest-order does not allow to add two order marks. However I came up with some solution for you.

You can resolve this syntax using pytest_generate_tests pytest hook. Just add it to your conftest.py file.

Example below will read all pytest.mark.order marks and make parametrized test out of it (if more than 1 order mark was provided). It adds parameter called order which stores arguments specified in pytest.mark.order.

conftest.py

def _get_mark_description(mark):
    if mark.kwargs:
        return ", ".join([f"{k}={v}" for k, v in mark.kwargs.items()])
    elif mark.args:
        return f"index={mark.args[0]}"
    return mark


def pytest_generate_tests(metafunc):
    """
    Handle multiple pytest.mark.order decorators.

    Make parametrized tests with corresponding order marks.
    """
    if getattr(metafunc, "function", False):
        if getattr(metafunc.function, "pytestmark", False):
            # Get list of order marks
            marks = metafunc.function.pytestmark
            order_marks = [
                mark for mark in marks if mark.name == "order"
            ]
            if len(order_marks) > 1:
                # Remove all order marks
                metafunc.function.pytestmark = [
                    mark for mark in marks if mark.name != "order"
                ]
                # Prepare arguments for parametrization with order marks
                args = [
                    pytest.param(_get_mark_description(mark), marks=[mark])
                    for mark in order_marks
                ]
                if "order" not in metafunc.fixturenames:
                    metafunc.fixturenames.append("order")
                metafunc.parametrize('order', args)

test_order.py

import pytest


@pytest.mark.order(6)
@pytest.mark.order(4)
def test_4_and_6():
    pass


@pytest.mark.order(5)
@pytest.mark.order(3)
@pytest.mark.order(1)
def test_1_3_and_5():
    pass


@pytest.mark.order(2)
def test_2():
    pass

pytest test_order.py -v Output

collecting ... collected 6 items

test_order.py::test_1_3_and_5[index=1] 
test_order.py::test_2 
test_order.py::test_1_3_and_5[index=3] 
test_order.py::test_4_and_6[index=4] 
test_order.py::test_1_3_and_5[index=5] 
test_order.py::test_4_and_6[index=6]

As you can see, all tests ran in defined order.

UPD

I have updated hook to make it compatible with other features of pytest-order. Also I have created PR in pytest-order GitHub repo.

Very nice! This is simpler than I thought it would be - I wonder if we should include this functionality in `pytest-order`... — MrBean Bremen, Mar 02 '23 at 18:49
Actually I had this thought yesterday but it definitely needs some testing with every other feature of `pytest-order` since it has a lot of different stuff like order scopes etc. https://github.com/pytest-dev/pytest-order#features. Well, anyways I can make a PR, and we will see :) — pL3b, Mar 03 '23 at 10:15

Pytest order test A, then test B and then test A again

1 Answers1