0

Is it possible to distribute pytest test cases across multiple different computers in the same network (in a cluster)? I've been using pytest and the xdist plugin for quite some time and would like to scope out the possibility of adding an additional machine to execute some of the test cases. What would it take to do that?

It sounds far fetched, but perhaps it is possible to virtualise additional CPU cores in the operating system (Linux) which are provided by the additional machine(s) and pytest could use them as workers, just like the local CPU cores.

I found execnet a library that is used by pytest-xdist which seems to be providing an ability to distribute tasks over the network, but it's (a) in maintenance mode and (b) there's little documentation on how to use for the purpose of distributing test cases in a cluster (I believe it's only used internal to pytest-xdist).

Perhaps I'm too focused on pytest-xdist and other pytest extensions could help.

Any pointer in the right direction is appreciated.

orange
  • 7,755
  • 14
  • 75
  • 139
  • the pytest-xdist README has a number of distributed modes available: https://pypi.org/project/pytest-xdist/#distribution-modes. Did you see those? If those are not going to work, can you be more specific about what you've tried and what's not working? – Michael Delgado Aug 18 '22 at 00:02
  • @MichaelDelgado I saw those options, but believe they dispatch test cases across CPUs (or cores) or the same machine (not across a cluster which is what I am looking for). – orange Aug 18 '22 at 05:45
  • 2
    You can send tests over SSH or remote sockets with `xdist` to be executed on another machines. – hoefling Aug 18 '22 at 08:01
  • @hoefling That would require manual division of test cases into a batch that is remotely executed and in to another batch which is executed locally. Is there a way to have `xdist` distribute test cases itself? – orange Aug 22 '22 at 01:28

0 Answers0