6

I have been using pytest-mock library for mocking with pytest. When I'm trying to run the test using tox command, I am getting the following error:

...
tests/test_cli.py ....EEEE
...
file /path/to/test_cli.py, line 63
  def test_cli_with_init_cmd_fails_with_db_error(runner, mocker, context):
E       fixture 'mocker' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, context, cov, doctest_namespace, fs, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, runner, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

However, when I try to run the test directly using pytest from within my venv, everything works as expected.

$ py.test --cov esmigrate --cov-report term-missing

...
platform linux -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /path/to/project/root, configfile: tox.ini
plugins: cov-2.10.1, pyfakefs-4.0.2, mock-3.3.1, requests-mock-1.8.0
collected 50 items                                                                                                                                                                                                

tests/test_cli.py ........                                         [ 16%]
tests/test_contexts/test_context_config.py ...                     [ 22%]
tests/test_internals/test_db_manager.py ..........                 [ 42%]
tests/test_internals/test_glob_loader.py .....                     [ 52%]
tests/test_internals/test_http_handler.py .......                  [ 66%]
tests/test_internals/test_script_parser.py .................       [100%]
...

Which is strange, because, I have added pytest-mock in my requirements.txt file, which was used to install dependencies within the venv, and I have this file added as a dependency for tox testenv as well. This is the content of my tox.ini file.

[tox]
envlist=py36, py37, py38, flake8

[pytest]
filterwarnings =
    error::DeprecationWarning
    error::PendingDeprecationWarning

[flake8]
max-line-length = 120
select = B,C,E,F,W,T4,B9,B950
ignore = E203,E266,E501,W503,D1

[testenv]
passenv=USERNAME
commands=py.test --cov esmigrate {posargs} --cov-report term-missing
deps= -rrequirements.txt

[testenv:flake8]
basepython = python3.8
deps =
    flake8
commands =
    flake8 esmigrate tests

A snapshot of requirements.txt file

...
pyfakefs==4.0.2
pyparsing==2.4.7
pyrsistent==0.17.3
pytest==6.1.1
pytest-cov==2.10.1
pytest-mock==3.3.1
PyYAML==5.3.1
...

This doesn't cause any problem when ran from travis-ci either, but I want to know what's the problem here and what I've been doing wrong. Was tox-env unable to install pytest-mock, or did "mocker" fixture got shadowed by something else?

Zobayer Hasan
  • 2,187
  • 6
  • 22
  • 38
  • 1
    Tox has difficulties seeing if the environment changed, so you often need to recreate the tox environment after changing dependencies. Travis starts blank and does not have this problem. So you need to run `tox` with the `--recreate` flag locally. – Shine Oct 19 '20 at 14:06
  • @Shine you might be onto something here. I deleted `.tox` directory and the issue went away. Looks like I can pass in `recreate=true` in `tox.ini`, but is it a good idea though? Or, should people just wait for the error to happen. – Zobayer Hasan Oct 19 '20 at 14:20
  • If you change any dependency in your requirements.txt it is normal to run 'tox --recreate'. In fact, you have to. – Shine Oct 20 '20 at 16:41
  • Just tested several times, For some reason I did not have to. In the console output I can see that it was installing the libraries. This is still all very strange. – Zobayer Hasan Oct 21 '20 at 17:40
  • So, this seems a bit inconsistent to me. – Zobayer Hasan Oct 21 '20 at 17:41

1 Answers1

4

tox currently (though this is planned to be improved in the (at the time of writing) current rewrite) does not recreate environments if files it does not manage change (such as requirements.txt / setup.py)

For a related question, you can see my question and workarounds

the core issue here is if you're not managing tox environment dependencies directly inline in tox.ini it will not notice changes (such as adding / removing dependencies from requirements.txt) and so you will need to run tox with the --recreate flag to reflect those changes

disclaimer: I'm one of the current tox maintainers

anthony sottile
  • 61,815
  • 15
  • 148
  • 207
  • I ended up using `--recreate` flags. Since I use some shell scripts to run tox tests, I think creating tox env every time is not a problem. Much easier than writing intelligent script to notice changes in requirements files. Thanks. – Zobayer Hasan Nov 25 '20 at 03:42