15

My pytest setup is running very slow, especially during collection phase.

So I have put up a pytest setup for my Django project, and each Django app's test-files are located in its own folder, i.e., the tree looks as follows

root
|
+--a
|  |
|  +--tests
|       |
|       +--conftest.py
|       +--testAa.py
|       +--testBa.py
+--b
|  |
|  +--tests
|       |
|       +--conftest.py
|       +--testAb.py
|       +--testBb.py
...
pytest.ini

The pytest.ini file specifies where to look for the tests and has the following content

[pytest]
DJANGO_SETTINGS_MODULE = project.project.settings.test
python_files = tests.py test_*.py *_tests.py
addopts = --reuse-db

For each app within the tests folder I have a file called contest.py. This file creates a set of objects that are used multiple times in the test files. For instance if an object of class A is used more than once, a contest creates that variable once and the tests use this conftest as input. Each conftest has the decorator @pytest.fixture(scope="function"), and the tests have the decorator @pytest.mark.django_db.

I don't think the loading times are caused by the conftests, or the decorators discussed in the last paragraph, but rather the tree structure and pytest.ini file I have put up. Are there any rules for what is a good structure for this? As I said, the loading times are extremely high for collecting the tests. To be more accurate, I have about 80 tests, and collecting them takes about 40 seconds. Running them all takes an additional 20.

sanyassh
  • 8,100
  • 13
  • 36
  • 70
Sindre Bakke Øyen
  • 175
  • 1
  • 1
  • 6
  • Is it still slow when restricting the directory to scan? E.g. `pytest --collect-only root/a/tests`. Maybe you have some large dir in project root which slows down the scan. If this is the case, you can restrict the directories via `norecursedirs` or `testpaths` options. – hoefling Apr 25 '19 at 19:59
  • It runs in 0.03 seconds in that case. I will try your second comment tomorrow, but it's getting late now. Thanks :) – Sindre Bakke Øyen Apr 25 '19 at 20:14
  • @hoefling Okay, I couldn't hold it, I had to check. Using norecursedirs was an amazing tip. I had lots of static and media-files in two separate folders (static/ and media/) and they both had tons of files to look through. If you post an answer I will accept it. – Sindre Bakke Øyen Apr 25 '19 at 20:34

3 Answers3

24

More or less restating my other answer:

When you invoke pytest, it will scan every subdirectory in project root, looking for tests. This may slow down the test collection; it may be wise to exclude unrelated directories from being scanned. pytest offers two configuration options for that:

  1. norecursedirs - holds directories that will be excluded from scanning. Use this option when you are looking for the pattern "include all, exclude selected". By default, norecursedirs is set to '.*', 'build', 'dist', 'CVS', '_darcs', '{arch}', '*.egg', so beware that when you override this option, the defaults are gone and you have to add them back.
  2. testpaths - holds directories that should only be considered for the scan, so this is basically the opposite to what norecursedirs is doing. Use this option when looking for the pattern "exclude all, include selected". This option also adds some minor or more significant speedup to the test discovery, depending on what you keep in the project root - most of the sudbirectories won't be traversed at all and the tests run starts sooner.

Usage: either place the options in the pytest.ini/setup.cfg/tox.ini:

[tool:pytest]
testpaths = tests othertests doc

or pass them via --override-ini from command line.

pytest -o "testpaths=tests othertests doc" ...
hoefling
  • 59,418
  • 12
  • 147
  • 194
  • How about mixing `testpaths` with `norecursedirs`? Would it work? – Blademaster Apr 07 '21 at 21:48
  • The syntax varies slightly depending on which of the config files you use. For `pytest.ini`, it is `[pytest]`, not `[tool.pytest]`. See https://docs.pytest.org/en/6.2.x/customize.html. Also, I could not get it to work using `pyproject.toml`. – h1-the-swan Feb 14 '22 at 18:40
8

For me the issue was with a particular conftest.py taking a very long time to run (it was setting up some huge fixtures for a different test that I didn't need here).

This was tricky to diagnose, since the delay still occured with --collect-only flag and custom rootdir set to the test directory only (the conftest was in a different directory far away but was still being detected somehow).

The solution for me was to run pytest --noconftest.

maxymoo
  • 35,286
  • 11
  • 92
  • 119
  • I can second this...I changed the way some of my signals worked and it threw off testing quite a bit because I expected items to be in the DB at the right time...due to that a simple fix was to add some extra steps to my base fixtures in conftest.py and that doubled my testing time as it was hitting the DB hard for each of the tests I had in the project – ViaTech Mar 08 '23 at 21:02
1

This is a top result when googling about Pytest slowness, so posting my experience.

In my case, collection was fast but running the first test was very slow. I fixed it by telling Pytest to reuse the existing test database (--reuse-db) and skipping migrations (--nomigrations).

This, combined with pytest-watch, makes TDD a breeze.

djanowski
  • 5,610
  • 1
  • 27
  • 17