How to run tox -e myenv against other environments?

Question

Is it possible in tox to have tox -e hello trigger the hello environment and run on the global envlist?

Running tox runs correctly against py2 and py3.

Running tox -e hello only runs against py3.

[tox]
skipsdist = True
envlist = py{27,37}

[testenv]
deps =
    pytest
    !py27: mock
commands =
    {envpython} --version

; How to run this hello against the complete envlist?
[testenv:hello]
envlist = py{27,37}
commands =
    echo hello

It seems this is expected behavior in some way, but I can't figure out how to use tox to solve the problem apart from making a bunch of tox files: I want to have different command entry points to tox. For instance, just run the unit tests against py27, just run the unit tests against py{27,37}, just run the integration tests against py27-{libA,libB}, etc.

Maybe it's just an anti pattern that you ~can use tox -e this way for a single environment, but it breaks envlist?

Do I just have to make another tox.ini then? That seems very clumsy given how flexible and re-use oriented everything else is about tox. I feel like I'm missing something.

Oliver Bestwalter · Accepted Answer · 2019-05-31T23:21:05.387

There are two main things to understand here:

envlist is a global setting in the [tox] section. Defining it in [testenv:hello] has no effect
envlist defines or generates a list of env names. This can be done either statically by providing a simple comma separated list or - like you did - generating the list combinatorically from factors provided via special syntax {...}. The result in both cases is a list of environment names that will then be executed in separated virtualenvs / venvs.

So to make it perfectly clear: in the end an envlist is a simple list of env names - nothing else. What is being done with those names (or their parts (a.k.a. factors)) is happening during execution in envs.

All of these these environments are executed if you call tox without -e. In your example you are generating an envlist with the {<factorX>,<factorY>} syntax which is based on the same principle like provided by commandline shells. If you ask for the generated environments in your tox config the result will be:

$ tox -a

py27-hello
py37-hello
hello

This might show already that things are a bit off in your config, because I don't think you want an unqualified hello env.

So what you are doing by generatively creating the envlist and statically providing the hello testenv using the [testenv:hello] syntax is mixing those in a way that then doesn't yield the result you would like to have.

So let's adapt your tox.ini to get rid of that unnecessary statically defined testenv:hello. We also get rid of the local envlist key as it had no effect anyway. It did not throw an error because arbitrarily named keys in envs are explicitly allowed and sometimes useful.

[tox]
skipsdist = True
envlist = py{27,37}-hello

[testenv]
deps =
    pytest
    !py27: mock
# do not warn that echo is a command not in the venv
whitelist_externals = echo
commands =
    {envpython} --version
    hello: echo hello

To make sure that the hello command is only run, when the env has a hello factor you can use <factor>[,<factor>]: <some command> to specify what should be run if the env name contains a certain factor (which are name parts separated by -) is generating a list of environment names that then serve as information for the concrete environments being executed.

If I call tox -a again on the changed ini I get:

py27-hello
py37-hello

Now I can ask for complete envs or only for factors by calling e.g. tox -e py27-hello or even tox -e hello which will then use the basepython and do everything that fits the factor.

To get more insight what is going on, you can display the completely resolved configuration for all environments, wich looks somethin like this:

$ tox --showconfig

[testenv:py27-hello]
  envdir          = /home/ob/do/play/.tox/py27-hello
  setenv          = SetenvDict: {'PYTHONHASHSEED': '1602746531', 'TOX_ENV_NAME': 'py27-hello', 'TOX_ENV_DIR': '/home/ob/do/play/.tox/py27-hello'}
  basepython      = python2.7
  description     = 
  envtmpdir       = /home/ob/do/play/.tox/py27-hello/tmp
  envlogdir       = /home/ob/do/play/.tox/py27-hello/log
  downloadcache   = None
  changedir       = /home/ob/do/play
  args_are_paths  = True
  skip_install    = False
  ignore_errors   = False
  recreate        = False
  passenv         = {'TOX_REPORTER_TIMESTAMP', 'TOX_WORK_DIR', 'TMPDIR', 'TOX_PARALLEL_ENV', 'LANGUAGE', 'LANG', 'PATH', 'LD_LIBRARY_PATH', 'PIP_INDEX_URL'}
  whitelist_externals = []
  platform        = .*
  sitepackages    = False
  alwayscopy      = False
  pip_pre         = False
  usedevelop      = False
  install_command = ['python', '-m', 'pip', 'install', '{opts}', '{packages}']
  list_dependencies_command = ['python', '-m', 'pip', 'freeze']
  deps            = [pytest]
  commands        = [['/home/ob/do/play/.tox/py27-hello/bin/python', '--version'], ['echo', 'hello']]
  commands_pre    = []
  commands_post   = []
  ignore_outcome  = False
  extras          = []
  depends         = ()
  parallel_show_output = False
[testenv:py37-hello]
  envdir          = /home/ob/do/play/.tox/py37-hello
  setenv          = SetenvDict: {'PYTHONHASHSEED': '1602746531', 'TOX_ENV_NAME': 'py37-hello', 'TOX_ENV_DIR': '/home/ob/do/play/.tox/py37-hello'}
  basepython      = python3.7
  description     = 
  envtmpdir       = /home/ob/do/play/.tox/py37-hello/tmp
  envlogdir       = /home/ob/do/play/.tox/py37-hello/log
  downloadcache   = None
  changedir       = /home/ob/do/play
  args_are_paths  = True
  skip_install    = False
  ignore_errors   = False
  recreate        = False
  passenv         = {'TOX_REPORTER_TIMESTAMP', 'TOX_WORK_DIR', 'TMPDIR', 'TOX_PARALLEL_ENV', 'LANGUAGE', 'LANG', 'PATH', 'LD_LIBRARY_PATH', 'PIP_INDEX_URL'}
  whitelist_externals = []
  platform        = .*
  sitepackages    = False
  alwayscopy      = False
  pip_pre         = False
  usedevelop      = False
  install_command = ['python', '-m', 'pip', 'install', '{opts}', '{packages}']
  list_dependencies_command = ['python', '-m', 'pip', 'freeze']
  deps            = [pytest, mock]
  commands        = [['/home/ob/do/play/.tox/py37-hello/bin/python', '--version'], ['echo', 'hello']]
  commands_pre    = []
  commands_post   = []
  ignore_outcome  = False
  extras          = []
  depends         = ()
  parallel_show_output = False
[testenv:hello]
  envdir          = /home/ob/do/play/.tox/hello
  setenv          = SetenvDict: {'PYTHONHASHSEED': '1602746531', 'TOX_ENV_NAME': 'hello', 'TOX_ENV_DIR': '/home/ob/do/play/.tox/hello'}
  basepython      = /usr/bin/python
  description     = 
  envtmpdir       = /home/ob/do/play/.tox/hello/tmp
  envlogdir       = /home/ob/do/play/.tox/hello/log
  downloadcache   = None
  changedir       = /home/ob/do/play
  args_are_paths  = True
  skip_install    = False
  ignore_errors   = False
  recreate        = False
  passenv         = {'TOX_REPORTER_TIMESTAMP', 'TOX_WORK_DIR', 'TMPDIR', 'TOX_PARALLEL_ENV', 'LANGUAGE', 'LANG', 'PATH', 'LD_LIBRARY_PATH', 'PIP_INDEX_URL'}
  whitelist_externals = []
  platform        = .*
  sitepackages    = False
  alwayscopy      = False
  pip_pre         = False
  usedevelop      = False
  install_command = ['python', '-m', 'pip', 'install', '{opts}', '{packages}']
  list_dependencies_command = ['python', '-m', 'pip', 'freeze']
  deps            = [pytest, mock]
  commands        = [['/home/ob/do/play/.tox/hello/bin/python', '--version'], ['echo', 'hello']]
  commands_pre    = []
  commands_post   = []
  ignore_outcome  = False
  extras          = []
  depends         = ()
  parallel_show_output = False

This is perfect and what I needed!! The `hello: echo hello` part is the critical piece I was fumbling for and the correct way to do it. And thanks for the showconfig, that is very helpful!! — SwimBikeRun, May 18 '19 at 19:57

How to run tox -e myenv against other environments?

1 Answers1