Make pytest require code coverage only on full test suite run

Question

I am using something like

# .coveragerc
fail_under = 100

and

# pytest.ini
[pytest]
addopts = --cov=modname/ --cov-report=term-missing

to make it so that my test suite runs the coverage and fails if it isn't 100%.

This works, but the problem is that if I run only a subset of the tests, like

pytest some/specific/test.py

it then complains that the coverage is not 100%, because of course that one single test file doesn't cover the entire codebase. Is there a better way to make pytest run coverage, but only when running the full test suite?

score 2 · Answer 1 · answered Jun 05 '20 at 13:00

You can temporarily override your .coveragerc by adding the following flag to your command:

--cov-fail-under=x

where x is the percentage to fail under (if you set this to 0, it will never fail based on the code coverage)

Thus, in your case you would run:

pytest some/specific/test.py --cov-fail-under=x

score 0 · Accepted Answer · answered Oct 27 '22 at 22:51

Using a tip from https://github.com/pytest-dev/pytest-cov/issues/418#issuecomment-657219659, I came up with the following (in conftest.py):

def pytest_configure(config):
    if config.args not in [[], ['mylib'], ['mylib/']]:
        cov = config.pluginmanager.get_plugin('_cov')
        cov.options.no_cov = True
        if cov.cov_controller:
            cov.cov_controller.pause()

This also has the benefit of actually disabling coverage, not just the reporting, which makes the tests run faster in these cases.

Make pytest require code coverage only on full test suite run

2 Answers2