73

According to Wikipedia and various articles it is best practice to divide tests into Unit tests (run first) and Integration tests (run second), where Unit tests are typically very fast and should be run with every build in a CI environment, however Integration tests take longer to run and should be more of a daily run.

Is there a way to divide these in pytest? Most projects don't seem to have multiple test folders, so is there a way to make sure I only run Unit, Integration or both according to the situtation (CI vs daily builds)? When calculating test coverage, I assume I will have to run both.

Am I going about this the right way in attempting to divide the tests into these categories? Is there a good example somewhere of a project that has done this?

Tom Malkin
  • 2,134
  • 2
  • 19
  • 35

3 Answers3

81

Yes, you can mark tests with the pytest.mark decorator.

Example:

def unit_test_1():
    # assert here

def unit_test_2():
    # assert here

@pytest.mark.integtest
def integration_test():
    # assert here

Now, from the command line, you can run pytest -m "not integtest" for only the unit tests, pytest -m integtest for only the integration test and plain pytest for all.

(You can also decorate your unit tests with pytest.mark.unit if you want, but I find that slightly tedious/verbose)

See the documentation for more information.

gmds
  • 19,325
  • 4
  • 32
  • 58
  • 1
    Ah this looks perfect Marcus! Do you agree that dividing the tests into unit and integration is a good idea for a python project using pytest? Is that what you do? – Tom Malkin Feb 27 '19 at 05:39
  • 1
    @Harlekuin if you mean in terms of metadata allowing them to be run separately, certainly; I too face the problem of larger-scale tests (and benchmarking) taking really long to run. If you mean in terms of directories, I would say that really depends on the project and the nature of the test. Hope I helped! – gmds Feb 28 '19 at 01:33
  • You definitely helped Marcus, I really appreciate it – Tom Malkin Feb 28 '19 at 02:46
  • 3
    Note that custom markers such as "integtest" should be registered into a pytest.ini file (see [documentation](https://docs.pytest.org/en/stable/example/markers.html#registering-markers)) otherwise, a warning will be raised – yoyoog Dec 18 '20 at 04:01
  • 5
    It's not a good idea to have unit tests and integration tests in the same file. Ideally they should be in a separate folder structure (as mentioned in the other answer) – Harry Jan 12 '21 at 13:17
  • to suppress the warning @yoyoog mentions in the pyproject.toml file. add `[tool.pytest.ini_options] markers = ["integtest"]` (with newlines) – JeroenDV Jul 07 '23 at 13:31
80

You can also structurally separate unit and integration tests into specific directories. Here is a sample file structure from A. Shaw's article Getting Started With Testing in Python:

enter image description here

With a structural approach, you:

  1. do not need to manually mark various tests with attributes or @pytest.mark.
  2. are not limited to a specific test runner. See examples below.

Examples

Here we run various test runners on integration tests alone. See the sample project/ directory in the figure above.

With unittest from the standard library:

λ python -m unittest discover -s tests/integration

With nose:

λ nose tests/integration

With pytest:

λ pytest tests/integration

Explicitly invoke the test runner on the tests/ directory to run all tests:

λ pytest tests/

Alternatively, many test runners have an auto test-discovery mechanism that can find tests in sub-directories.

λ cd ..
λ pytest project/
pylang
  • 40,867
  • 14
  • 129
  • 121
  • 2
    Nice, I like the idea of not being bound to a specific test runner. Would most test runners be able to discover both unit and integration if you just passed the parent "tests" folder? – Tom Malkin Feb 27 '19 at 22:14
  • 1
    Possibly. I still use the older `nose` test runner, which can also handle this kind of discovery. Consult the docs per library, as the command-line invocation will vary. – pylang Feb 27 '19 at 22:25
  • 1
    I can now confirm `pytest` also has [auto test-discovery](https://docs.pytest.org/en/latest/goodpractices.html). – pylang May 20 '19 at 17:07
  • 8
    When separated like this, you can also run `pytest tests/unit tests/integration` to execute the integration tests after the unit tests. – Doopy Apr 26 '20 at 21:47
  • 2
    In a case like this, how would you import my_app/helloworld.py inside tests/unit/ test_helloworld. I'm trying to follow this directory structure but imports can't seem to resolve for me. – Rose Nov 27 '20 at 17:12
  • you cannot get test coverage from both – Ievgen Jun 28 '21 at 17:10
  • @levgen toss them under "tox" and change the '--coverage' arguments to generate coverage for both. – joebeeson Dec 30 '21 at 17:35
  • @Rose Very good question. See [this answer](https://stackoverflow.com/a/73166679/4865723) which explains how to structure your project with the `src`-Layout and how to "install" it in _Development mode_ which makes you able to run the tests like described here. – buhtz Oct 17 '22 at 09:13
0

As of 2022 another option is the pytest-integration package, which adds some additional functionality as compared to using @pytest.mark decorators alone.

Oliver Coleman
  • 734
  • 7
  • 13