28

I wanted to run parameterized test functions in parallel. This is for a concurrency testing scenario. Same testcase runs in parallel with different parameters in a device. After completing all the parameterized variant of one test function, I want to proceed with the next one.

If we take this simple example, I want to run all 4 instances of test_even parallely and then move to test_odd.

@pytest.mark.parametrize("x", range(4))
def test_even(x):
    assert x % 2 == 0        
@pytest.mark.parametrize("x", range(4))
def test_odd(x):
    assert x % 2 != 0

Is it possible to do in pytest? I checked xdist, but could not find this type of support. Could anybody please give some pointers on how to implement this in pytest?

kanak
  • 533
  • 1
  • 5
  • 8
  • 1
    I don't think xdist will allow you to synchronize between tests (it will run all your tests on all available processes -- which is generally what you'd want) – thebjorn Nov 24 '16 at 11:56
  • The directly relevant GitHub issue: [Parallelization of the parameters #58](https://github.com/pytest-dev/pytest-xdist/issues/58), seems to be an open question as of Feb 18, 2019 – jxramos Feb 18 '19 at 18:45
  • It is possible to create your own xdist scheduler (you can fork an existing one and modify it). Then, you can schedule the tests per worker according to your logic. If you elaborate a little on what exactly you are trying to accomplish, it may be possible to think of an alternative way to accomplish it without modifying the scheduler. – Meir Jan 30 '22 at 19:14

1 Answers1

4

I'd recommend using xdist, from their website it does:

The pytest-xdist plugin extends pytest with new test execution modes, the most used being distributing tests across multiple CPUs to speed up test execution...

It does also parallelize your parametrized tests by default.

Here's an example:

import pytest

@pytest.mark.parametrize("foo", range(10))
def test_bar(foo):
    assert True

pytest -n 2 -v
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0 -- /tmp/test/venv/bin/python3
cachedir: .pytest_cache
rootdir: /tmp/test
plugins: xdist-2.5.0, forked-1.4.0
[gw0] linux Python 3.10.4 cwd: /tmp/test
[gw1] linux Python 3.10.4 cwd: /tmp/test
[gw0] Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
[gw1] Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
gw0 [10] / gw1 [10]
scheduling tests via LoadScheduling

test/test_foo.py::test_bar[1] 
test/test_foo.py::test_bar[0] 
[gw1] [ 10%] PASSED test/test_foo.py::test_bar[1] 
[gw0] [ 20%] PASSED test/test_foo.py::test_bar[0] 
test/test_foo.py::test_bar[3] 
[gw1] [ 30%] PASSED test/test_foo.py::test_bar[3] 
test/test_foo.py::test_bar[2] 
test/test_foo.py::test_bar[4] 
[gw0] [ 40%] PASSED test/test_foo.py::test_bar[2] 
[gw1] [ 50%] PASSED test/test_foo.py::test_bar[4] 
test/test_foo.py::test_bar[6] 
test/test_foo.py::test_bar[5] 
[gw1] [ 60%] PASSED test/test_foo.py::test_bar[6] 
[gw0] [ 70%] PASSED test/test_foo.py::test_bar[5] 
test/test_foo.py::test_bar[8] 
test/test_foo.py::test_bar[7] 
[gw1] [ 80%] PASSED test/test_foo.py::test_bar[8] 
test/test_foo.py::test_bar[9] 
[gw0] [ 90%] PASSED test/test_foo.py::test_bar[7] 
[gw1] [100%] PASSED test/test_foo.py::test_bar[9] 

================================================================================================ 10 passed in 0.28s =================================================================================================
Bastian Venthur
  • 12,515
  • 5
  • 44
  • 78