2

This is my tox.ini file:

# Tox (https://tox.readthedocs.io/) is a tool for running tests
# in multiple virtualenvs. This configuration file will run the
# test suite on all supported python versions. To use it, "pip install tox"
# and then run "tox" from this directory.
#
# See also https://tox.readthedocs.io/en/latest/config.html for more
# configuration options.

[tox]
# Choose your Python versions. They have to be available
# on the system the tests are run on.
# skipsdist=True
ignore_basepython_conflict=false

[testenv:{setup,lint,codestyle,docstyle,tests,doc-linux,doc-darwin,doc-win32}]
basepython=python3.9

envdir = {toxworkdir}/py39

setenv =
    PROJECT_NAME = project_name

passenv =
    WINDIR

install_command=
    pip install \
        --find-links=pkg \
        --trusted-host=pypi.python.org \
        --trusted-host=pypi.org \
        --trusted-host=files.pythonhosted.org \
        {opts} {packages}

platform = doc-linux: linux
           doc-darwin: darwin
           doc-win32: win32

deps =
    -r{toxinidir}/requirements-dev.txt
    -r{toxinidir}/requirements.txt

commands = 
    setup: python -c "print('All SetUp')"
    # Mind the gap, use a backslash :)
    lint: pylint -f parseable -r n --disable duplicate-code \
    lint:    --extension-pkg-whitelist=PyQt5,numpy,torch,cv2,boto3 \
    lint:    --ignored-modules=PyQt5,numpy,torch,cv2,boto3 \
    lint:    --ignored-classes=PyQt5,numpy,torch,cv2,boto3 \
    lint:    project_name \
    lint:    {toxinidir}/script

    lint: pylint -f parseable -r n --disable duplicate-code \
    lint:    demo/demo_file.py

    codestyle: pycodestyle --max-line-length=100 \
    codestyle:     --exclude=project_name/third_party/* \
    codestyle:     project_name demo script

    docstyle: pydocstyle \
    docstyle:     --match-dir='^((?!(third_party|deprecated)).)*' \
    docstyle:     project_name demo script

    doc-linux: make -C {toxinidir}/doc html
    doc-darwin: make -C {toxinidir}/doc html
    doc-win32: {toxinidir}/doc/make.bat html

    tests: python -m pytest -v -s --cov-report xml --durations=10 \
    tests:     --cov=project_name --cov=script \
    tests:     {toxinidir}/test
    tests: coverage report -m --fail-under 100

On tox<4.0 it was very convinient to run tox -e lint to fix linting stuff or tox -e codestyle tox fix codestyle stuff, etc. But now, with version tox>4.0 each time I run one of these commands I get this message (for instance):

codestyle: recreate env because env type changed from {'name': 'lint', 'type': 'VirtualEnvRunner'} to {'name': 'codestyle', 'type': 'VirtualEnvRunner'}
codestyle: remove tox env folder .tox/py39

And it takes forever to run these commands since the evironments are recreated each time ...

I also use these structure for running tests on jenkins so I can map each of these commands to a jenkins stage.

How can I reuse the environment? I have read that it is possible to do it using plugins, but no idea how this can be done, or how to install/use plugins.

I have tried this: tox multiple tests, re-using tox environment

But it does not work in my case.

I spect to reuse the environment for each of the environments defined in the tox file.

Marcus
  • 275
  • 1
  • 12

2 Answers2

3

As an addition to N1ngu's excellent answer...

You could re-structure your tox.ini as following:

[tox]
...

[testenv]
<here goes all the common configuration>

[testenv:lint]
<here goes the lint specific configuration>

[testenv:codestyle]
...

And so on. This is a common setup.

While still the environments need to be created at least once, they won't get recreated on each invocation.

This all said, you could also have a look at https://pre-commit.com/ to run your linters, which is very common in the Python community.

Then you would have a tox.ini like the following...

[tox]
...

[testenv]
<here goes all the common configuration>

[testenv:lint]
deps = pre-commit
commands = pre-commit run --all-files

There is now a definite answer about re-use of environments in the faq: https://tox.wiki/en/latest/upgrading.html#re-use-of-environments

Stephen Ostermiller
  • 23,933
  • 14
  • 88
  • 109
Jürgen Gmach
  • 5,366
  • 3
  • 20
  • 37
  • This is much more common and leaner approach. The biggest issue was virtualenvs couldn't be safely cached between CI jobs in case a recreation would be in order but... Tox 4 solves this!! The only issue left is some bloat in your local hard drive, not a big issue IMHO. – N1ngu Dec 28 '22 at 11:34
  • This is much better but if you have a lot of dependencies of huge libraries like torch, opencv, matplotlib, pandas, tensorflow, pil, etc it takes forever ... to do CI. – Marcus Dec 28 '22 at 12:15
1

I fear the generative names + factor-specific commands solution you linked relied on tox-3 not auto-recreating the environments by default, which is among the new features in tox 4. Now, environment recreation is something that can be forced (--recreate) but can't be opted-out.

Official answer on this https://github.com/tox-dev/tox/issues/425 boils down to

Officially we don't allow sharing tox environments at the moment [...] As of today, each tox environment has to have it's own virtualenv even if the Python version and dependencies are identical [...] We'll not plan to support this. However, tox 4 allows one to do this via a plugin, so we'd encourage people [...] to try it [...]. Once the project is stable and widely used we can revisit accepting it in core.

So that's it, write a plugin. No idea on how to do that either, so my apologies if this turns out as "not an answer"

N1ngu
  • 2,862
  • 17
  • 35
  • 2
    So I guess that I have to find an alternative to tox. – Marcus Dec 28 '22 at 11:08
  • 2
    For applications, I dropped tox precisely because of this: a single pipenv virtualenv, cached between CI jobs and some scripts and Justfile/Makefile targets do a much better job. Yet, for libraries, tox is still the best to run tests on different interpreters and dependency sets (something e.g. pipenv would never do). – N1ngu Dec 28 '22 at 11:21
  • Yes, I think I will do that ... – Marcus Dec 28 '22 at 12:17