Use docstrings to list tests in py.test

Question

Here is a simple test file:

# test_single.py
def test_addition():
    "Two plus two is still four"
    assert 2 + 2 == 4

def test_addition2():
    "One plus one is still two"
    assert 1 + 1 == 2

The default output in py.test is like

$ py.test test_single.py -v
[...]
test_single.py::test_addition PASSED
test_single.py::test_addition2 PASSED

I would like to have

Two plus two is still four PASSED
One plus one is still two PASSED

i.e. use the docstrings as descriptions for the tests.

I tried to use a customization in a conftest.py file:

import pytest

@pytest.mark.tryfirst
def pytest_runtest_makereport(item, call, __multicall__):
    # execute all other hooks to obtain the report object
    rep = __multicall__.execute()
    if rep.when == "call":
        extra = item._obj.__doc__.strip()
        rep.nodeid =  extra
    return rep

that is close, but it repeats the filename on every line:

$ py.test test_single.py
======================================================================================== test session starts =========================================================================================
platform darwin -- Python 2.7.7 -- py-1.4.26 -- pytest-2.6.4
plugins: greendots, osxnotify, pycharm
collected 2 items

test_single.py
And two plus two is still four .
test_single.py
And one plus one is still two .

====================================================================================== 2 passed in 0.11 seconds ======================================================================================

How can I avoid the lines with test_single.py in the output, or maybe print it only once?

Looking into the source of py.test and some of its plugins did not help.

I am aware of the pytest-spec plugin, but that uses the function's name as a description. I don't want to write def test_two_plus_two_is_four().

Answer 1

To expand on my comment to @michael-wan's answer: to achive something similar to specplugin put into conftest.py:

def pytest_itemcollected(item):
    par = item.parent.obj
    node = item.obj
    pref = par.__doc__.strip() if par.__doc__ else par.__class__.__name__
    suf = node.__doc__.strip() if node.__doc__ else node.__name__
    if pref or suf:
        item._nodeid = ' '.join((pref, suf))

and the pytest output of

class TestSomething:
"""Something"""

def test_ok(self):
    """should be ok"""
    pass

will look like

If you omit docstrings class/func names will be used.

Answer 2

I was missing rspec in ruby for python. So, based on the plugin pytest-testdox., I have written similar one which takes doc strings as report message. You can check it out pytest-pspec.

Answer 3

For a plugin that (I think) does what you want out of the box, check out pytest-testdox.

It provides a friendly formatted list of each test function name, with test_ stripped, and underscores replaced with spaces, so that the test names are readible. It also breaks up the sections by test file.

This is what the output looks like:

Answer 4

@Matthias Berth, you can try to use pytest_itemcollected

def pytest_itemcollected(item):
""" we just collected a test item. """
    item.setNodeid('' if item._obj.__doc__ is None else item._obj.__doc__.strip() )

and modify pydir/Lib/site-packages/pytest-2.9.1-py2.7.egg/_pytest/unittest.py add the following function to the TestCaseFunction class

def setNodeid(self, value):
    self._nodeid = value

and the result will be :

platform win32 -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- D:\Python27\python.exe
cachedir: .cache
rootdir: E:\workspace\satp2\atest\testcase\Search\grp_sp, inifile:
plugins: html-1.8.0, pep8-1.0.6
collecting 0 itemsNone
collected 2 items
Two plus two is still four <- sut_amap3.py PASSED
One plus one is still two <- sut_amap3.py PASSED

by the way when you are using pytest-html you can use the pytest_runtest_makereport function you make and it will generate the report with the name you customized. hope this helps.

Answer 5

I wanted to do the same but in a simpler way, preferably without an external plugin to do more than needed and also avoiding changing the nodeid as it could break other things

I came up with the following solution:

test_one.py

import logging

logger = logging.getLogger(__name__)

def test_one():
    """ The First test does something """
    logger.info("One")

def test_two():
    """ Now this Second test tests other things """
    logger.info("Two")

def test_third():
    """ Third test is basically checking crazy stuff """
    logger.info("Three")

conftest.py

import pytest
import inspect

@pytest.mark.trylast
def pytest_configure(config):
    terminal_reporter = config.pluginmanager.getplugin('terminalreporter')
    config.pluginmanager.register(TestDescriptionPlugin(terminal_reporter), 'testdescription')

class TestDescriptionPlugin:

    def __init__(self, terminal_reporter):
        self.terminal_reporter = terminal_reporter
        self.desc = None

    def pytest_runtest_protocol(self, item):
        self.desc = inspect.getdoc(item.obj)

    @pytest.hookimpl(hookwrapper=True, tryfirst=True)
    def pytest_runtest_logstart(self, nodeid, location):
        if self.terminal_reporter.verbosity == 0:
            yield
        else:
            self.terminal_reporter.write('\n')
            yield
            if self.desc:
                    self.terminal_reporter.write(f'\n{self.desc} ')

Running with --verbose

============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.1.dev62+g2d9dac95e, py-1.8.1, pluggy-0.13.1 -- C:\Users\Victor\PycharmProjects\pytest\venv\Scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\Victor\PycharmProjects\pytest, inifile: tox.ini
collecting ... collected 3 items


test_one.py::test_one 
The First test does something  PASSED                                    [ 33%]

test_one.py::test_two 
Now this Second test tests other things  PASSED                          [ 66%]

test_one.py::test_third 
Third test is basically checking crazy stuff  PASSED                     [100%]

============================== 3 passed in 0.07s ==============================

Running with --log-cli-level=INFO

============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.1.dev62+g2d9dac95e, py-1.8.1, pluggy-0.13.1
rootdir: C:\Users\Victor\PycharmProjects\pytest, inifile: tox.ini
collected 3 items


test_one.py::test_one 
The First test does something  
-------------------------------- live log call --------------------------------
INFO     test_one:test_one.py:7 One
PASSED                                                                   [ 33%]

test_one.py::test_two 
Now this Second test tests other things  
-------------------------------- live log call --------------------------------
INFO     test_one:test_one.py:11 Two
PASSED                                                                   [ 66%]

test_one.py::test_third 
Third test is basically checking crazy stuff  
-------------------------------- live log call --------------------------------
INFO     test_one:test_one.py:15 Three
PASSED                                                                   [100%]

============================== 3 passed in 0.07s ==============================

The plugin in conftest.py is probably simple enough for anyone to customize according to their own needs.