For loop in unittest

Question

Is there a way to tell the python unittest execute all assertion in a method and show all cases it fails, instead of stop at the first failed.

class MyTestCase(TestCase):
    def test_a(self):
        with open('testcase.txt') as ifile: 
            for iline in ifile:
                self.assertEqual(iline, 'it is a test!')

I have yet to see a unit test framework that accumulates errors and reports them all. It seems that the standard is to fail on first failure. — TGH, Oct 20 '14 at 04:37
@TGH: I have; [googletest](https://code.google.com/p/googletest/wiki/Primer) has `EXPECT` macros that continue on failure and `ASSERT` macros that abort immediately. — user2357112, Oct 20 '14 at 04:44
@user2357112 Ok interesting. I guess it makes sense since it's just a design choice made by the creator of the framework. I guess a common pattern is to throw exceptions when asserts fail, leaving it to the caller to implement their own tally of exceptions if desirable. — TGH, Oct 20 '14 at 04:46

vaultah · Accepted Answer · 2015-02-25T19:24:39.400

23

Python 3.4 introduced the subTest context manager. Your code will look like

class MyTestCase(TestCase):
    def test_a(self):
        with open('testcase.txt') as ifile: 
            for iline in ifile:
                with self.subTest(line=iline):
                    self.assertEqual(iline, 'it is a test!')

The ugly way to achieve this without subTest is to make self.assert* calls within a try block, print the errors caught, and raise the AssertionError explicitly after the loop if at least one test failed.

edited Feb 25 '15 at 19:24

answered Oct 20 '14 at 04:38

vaultah

44,105
12
114
143

1

@lucemia subTest is unarguably the easiest way to achieve what you want, but you can also use `try..except` in a loop to print exceptions. – vaultah Oct 20 '14 at 05:41
2

The problem with subtests is that `setUp` is not automatically called for each subtest, as can sometimes be necessary. It can however certainly be called manually as `self.setUp()` within the subtest. – Asclepius May 01 '17 at 20:14

score 5 · Answer 2 · answered Feb 25 '15 at 19:08

Alternatively, you can make a data-driven-test with the help of ddt package:

DDT (Data-Driven Tests) allows you to multiply one test case by running it with different test data, and make it appear as multiple test cases.

import unittest
from ddt import ddt, data

@ddt
class FooTestCase(unittest.TestCase):
    @data('it is a test!', 'it is a test!', 'something else')
    def test_lines(self, value):
        self.assertEqual(value, 'it is a test!')

ddt can also have the data coming from a file, but it has to be a JSON file.

ddt-package [now supports Yaml-Files as well](https://ddt.readthedocs.io/en/latest/example.html) — bk_, Dec 28 '21 at 15:51

score 1 · Answer 3 · answered Oct 20 '14 at 04:38

1

Generally no. Unit tests stop at first fail. If you want to compare all lines you need to have a local list and then put the different lines into the list. Then assert that list length is zero. Or a local boolean variable.

answered Oct 20 '14 at 04:38

artm

8,554
3
26
43

For loop in unittest

3 Answers3