2

Is it possible to somehow see list of tests in the order in which they were performed using pytest --random-order flag? In random-order package documentation (https://pypi.org/project/pytest-random-order/) I found only that it reports seed so you can re-run tests in same order, but I need to see which test run before each other to find out some dependency (my tests are green in normal order, but red in some runs of random order) Thanks in advance.

Filip Jašek
  • 71
  • 9
  • 2
    Just use `pytest -v` to show the tests while re-running them with the same seed. – MrBean Bremen Mar 29 '21 at 15:09
  • Thanks, this can do the trick, but I have more than hundered tests with quite a lots of debug prints so it is hard to search for START of every single test (some are just out of scrolling area of my terminal), so I'd like to see only list of test in order they were performed... But again, if this is the only way, I'll try to use it, so thak you! – Filip Jašek Mar 29 '21 at 15:30
  • Pipe the output into a file, this will be easier to handle (for example by removing unwanted lines using a regex). – MrBean Bremen Mar 29 '21 at 16:51
  • 1
    As an aside: I meanwhile wrote a small [pytest plugin](https://pypi.org/project/pytest-find-dependencies/) which tries to find dependent tests automatically by running tests multiple times in different order. – MrBean Bremen Feb 13 '22 at 14:08

1 Answers1

4

I got an idea to look on this problem from pytest perspective not just random-order plugin and I find that all I needed to do is add flag -rA where A stands for All test passed, skipped and failed. (See Pytest docs or Stack Overflow answers here) So to run test in random order and with summary (list) of tests at the end of the run use:

pytest -rA --random-order
Filip Jašek
  • 71
  • 9