Is there a way to add metadata in py files for grouping tests?

Question

Lets say I have the following testcases in different files

TestOne.py {tags: One, Two}
TestTwo.py {tags: Two}
TestThree.py {tags: Three}

Each of which inherits from unittest.TestCase. Is there any ability in python to embed metadata information within these files, so that I can have a main.py script to search for those tags and execute only those testcases?

For Eg: If I want to execute testcases with {tags: Two} then only testcases TestOne.py and TestTwo.py should be executed.

score 3 · Accepted Answer · edited Apr 11 '13 at 07:10

The py.test testing framework has support for meta data, via what they call markers.

For py.test test cases are functions that have names starting with "test", and which are in modules with names starting with "test". The tests themselves are simple assert statements. py.test can also run tests for the unittest library, and IIRC Nose tests.

The meta data consists of dynamically generated decorators for the test functions. The decorators have the form: @pytest.mark.my_meta_name. You can choose anything for my_meta_name. There are a few predefined markers that you can see with py.test --markers.

Here is an adapted snippet from their documentation:

# content of test_server.py

import pytest

@pytest.mark.webtest
def test_send_http():
    pass # perform some webtest test for your app

def test_always_succeeds():
    assert 2 == 3 - 1

def test_will_always_fail():
    assert 4 == 5

You select marked tests with the -m command line option of the test runner. To selectively run test_send_http() you enter this into a shell:

py.test -v -m webtest

score 2 · Answer 2 · edited Apr 11 '13 at 06:42

Of course it's more easy to define tags in the main module, but if it's important for you to save them with test files, it could be a good solution to define it in test files like this:

In TestOne.py:

test_tags = ['One', 'Two']
...

Then you can read all tags in the initialize function of your main module in this way:

test_modules = ['TestOne', 'TestTwo', 'TestThree']
test_tags_dict = {}

def initialize():
    for module_name in test_modules:
        module = import_string(module)

        if hasattr(module, 'test_tags'):
            for tag in module.test_tags:
                if tag not in test_tags_dict:
                    test_tags_dict[tag] = []
                test_tags_dict[tag].append(module)

So you can implement a run_test_with_tag function to run all tests for an specific tag:

def run_test_with_tag(tag):
    for module in test_tags_dict.get(tag, []):
        # Run module tests here ...

Is there a way to add metadata in py files for grouping tests?

2 Answers2