0

I have a simple dockerfile that does the following:

FROM python:3.10.3

RUN pip install -U pip setuptools
RUN pip install poetry==1.1.11

WORKDIR /srv

I am using docker compose to mount the relevant directory, but will ignore that here. I am then running this command to run tests in 1 folder:

docker-compose run -it -e PYTHONPATH=./src api /bin/bash -c "poetry run pytest --pspec ./src/e2e/test_*"

That folder has 2 files that match the glob test_* and they contain 15 tests.

When I run the command above using the --collectonly flag, it takes about 75s.

Example output below:

===================================================== test session starts =====================================================
platform linux -- Python 3.10.3, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /srv
plugins: pspec-0.0.4, mock-3.10.0, anyio-3.6.2
collected 15 items

<Module src/e2e/test_m.py>
  <Function test_b_idempotency>
  <Function test_e_creation>
  <Function test_e_update>
  <Function test_v_creation>
  <Function test_v_update>
  <Function test_s_cleared>
<Module src/e2e/test_q.py>
  <Function test_basic_q>
  <Function test_basic_r>
  <Function test_basic_p>
  <Function test_basic_r2>
  <Function test_time_weight>
  <Function test_s_content>
  <Function test_simple_research_case>
  <Function test_context_e_search>
  <Function test_recent_m_pagination>

=========================================== 15 tests collected in 77.41s (0:01:17) ============================================

This used to be an incredibly quick command (<1s) but I am not entirely sure what has slowed it down apart from possibly an upgrade of docker.

The tests also take unusually long to run, however given collection is taking incredibly long, this seems like something outside of what's in the tests themselves.

I am on macOS 12.6 and Docker 4.15.0

Does anyone know what may be causing the issue, or how I may go about troubleshooting?

I have read the answers in this question but the answers revolve around reducing the amount of files being scanned, which I think I'm already doing.

Any help is much appreciated.

EDIT: This does not seem to be a docker issue since collection is also slow when I run locally.

rbhalla
  • 869
  • 8
  • 32
  • You should probably check what is in your workdir if you suspect collection is taking too long, swapping the command for`ls -Al` would be a good start. – ljmc Jan 08 '23 at 23:07
  • The working directory contains the entire codebase for this service, however the command specifics a specific folder's files `./src/e2e/test_*`, should it not be only looking at those files? – rbhalla Jan 08 '23 at 23:23
  • pytest will try to identify a `rootdir` using the file(s)/path(s) you give it, and its config file ([docs](https://docs.pytest.org/en/stable/reference/customize.html#initialization-determining-rootdir-and-configfile)). Please [edit] your question and add the output you're getting for your pytest run. – ljmc Jan 08 '23 at 23:36
  • Bind mounts on MacOS are known to be particularly slow, especially if you're trying to mount an entire application or virtual environment into a container. Your MacOS host comes with Python preinstalled; can you run your unit tests there, maybe in a virtual environment? – David Maze Jan 08 '23 at 23:55
  • I have included an example. Reading those docs, I also tried specifying `--rootdir=./src/e2e` however that did not speed things up unfortunately. – rbhalla Jan 08 '23 at 23:58
  • @DavidMaze, good call out. Interestingly, when I install all the dependencies locally and run the same command it takes about 60s. A small improvement, but not anywhere near how quickly I would expect it to be. My processor is a 2.3ghz i9 so I will hope it's not that bottlenecking somewhere. – rbhalla Jan 09 '23 at 00:16

0 Answers0