Why do assertions in unittest use TestCase.assertEqual not the assert keyword?

Question

Python's built-in unittest module makes assertions with TestCase.assert* methods:

class FooTest(TestCase):
    def test_foo(self):
        self.assertEqual(1,1)
        self.assertNotEqual(1,2)
        self.assertTrue(True)

I have generally used a testrunner such as nose or py.test which allow use of the built-in assert keyword when making assertions:

assert 1 == 1
assert 1 != 2
assert True

What is the motivation for unittest's TestCase.assert* approach and what are the strengths and weaknesses of this vs asserting with the built-in assert keyword? Are there reasons why unittest's syntax should be favoured?

This is a common idiom in testing frameworks, regardless of language. Nonetheless a good question. — Matt Ball, Jun 15 '11 at 16:39
Duplicate of [What are the advantages or difference in “assert False” and “self.assertFalse”](http://stackoverflow.com/q/2958169/95735) — Piotr Dobrogost, Oct 30 '12 at 22:07

score 41 · Accepted Answer · answered Jun 15 '11 at 16:40

41

The problem with the assert keyword is that it is optimized out, and thus ignored, when Python is run in 'optimized' mode (with the -O argument or with the PYTHONOPTIMIZE environment variable set.) If tests were to use assert then testing with -O would be impossible.

Additionally, the use of the assert methods makes it trivial to report about what the values involved actually were, without having to dig into the stack and the source and figure out what they were supposed to be (which, I believe, is the technique nose and py.test use for this.)

answered Jun 15 '11 at 16:40

Thomas Wouters

130,178
23
148
122

11

If you're testing code, you're probably not running it in optimized mode. – JBernardo Jun 15 '11 at 16:43
3

+1 although I doubt someone will run their unit tests using the `-O` switch. – Ferdinand Beyer Jun 15 '11 at 16:43
13

If you're not testing code the same way as you're running it in production, you're not doing it right. The testing framework isn't the thing to decide you shouldn't use `-O` in production (although you probably shouldn't.) – Thomas Wouters Jun 15 '11 at 16:44
1

Thanks @Thomas. If I understand correctly: while `nose` and `py.test` provide an alternative assertions to `unittest`, they come at the cost of compromising `-O` and relying on a more complex reporting implementation. – Stephen Emslie Jun 17 '11 at 13:43
4

JBernardo: Respectfully disagree -- ever have code that worked fine in Debug but acted funny in Release (or vice-verse)? It can happen and in most cases you should test in as close as final an environment as possible. – SilentDirge Mar 17 '14 at 02:38
1

This is a super bad answer. Plain and simple. Also it's technically wrong for pytest as their assertion rewriting will rewrite the asserts to not use the assert statement. – boxed Jan 13 '22 at 13:45

score 6 · Answer 2 · answered Jun 16 '11 at 10:40

I don't see a concrete design decision for this. Looking at the unittest docs it states that

the type specific equality function will be called in order to generate a more useful default error message

So I would say it is an implementation decision to help produce more meaningful errors etc.

Rabarberski · Answer 3 · 2023-02-03T09:47:26.643

Usage of specific test methods like TestCase.assertEqual(x,y) instead of assert x == y can result in more clear reporting of what is wrong.

For example, consider:

from unittest import TestCase

class AssertApproachesTests(TestCase):
    def test_plain_assert(self):
        assert {"key": "value"} == {"key": "valuesss"}

    def test_rich_assert(self):
        self.assertEqual({"key": "value"}, {"key": "valuesss"})

Running the tests (in this case, using pytest) gives:

$ pytest api -k "_assert"

======================== FAILURES =========================
_________ AssertApproachesTests.test_plain_assert _________

self = <api.tests.test_asserts.AssertApproachesTests testMethod=test_plain_assert>

    def test_plain_assert(self):
>       assert {"key": "value"} == {"key": "valuesss"}
E       AssertionError: assert {'key': 'value'} == {'key': 'valuesss'}
E         Differing items:
E         {'key': 'value'} != {'key': 'valuesss'}
E         Use -v to get more diff

api/tests/test_asserts.py:5: AssertionError
_________ AssertApproachesTests.test_rich_assert __________

self = <api.tests.test_asserts.AssertApproachesTests testMethod=test_rich_assert>

    def test_rich_assert(self):
>       self.assertEqual({"key": "value"}, {"key": "valuesss"})
E       AssertionError: {'key': 'value'} != {'key': 'valuesss'}
E       - {'key': 'value'}
E       + {'key': 'valuesss'}
E       ?               +++

api/tests/test_asserts.py:8: AssertionError

Notice the "+++" in the latter output, showing what actually differs.

score 0 · Answer 4 · answered Jun 15 '11 at 16:43

The main strength of this approach is providing several built in tests that are commonly performed so that one doesn't have to write them over and over again. Additionally, assertRaises lets you customize the exact behavior of the assert by throwing an exception.

score 0 · Answer 5 · answered Jan 13 '22 at 14:00

0

The unittest syntax is a workaround for a weakness in how Python has implemented assert. Pytest has another workaround with assertion rewriting that largely fixes this problem in a much better way. It's also BETTER at giving nice error messages for asserts than the assertFoo style asserts.

Use pytest and live a happier and more productive life :P

answered Jan 13 '22 at 14:00

boxed

3,895
2
24
26

Could you elaborate on the "weakness"? – Amin Shah Gilani Feb 11 '22 at 14:42
The assert doesn't collect any information on what failed. Like `assert a == b` will not show the value of `a` or `b` or that it was a comparison. Just the stack trace. – boxed Feb 12 '22 at 15:44

score 0 · Answer 6 · answered Jul 28 '22 at 16:35

I did some research about the comparison between the two. From what I've read, including the others' points, here are a few points of advocates of self.assertEqual :

Tests will be run even with -O (optimization)
Test messages are nicer looking, in older versions of python the test might halt and does not show the full stack message
Can provide a log message

Advantages of assert are:

flexible (for e.g., compare one dictionary is part of another dictionary: assert dict1 <= dict2 )
simple & shorter to read

In my opinion,

For optimization, I doubt we run it with -O, so self.assertEqual
I did a small test about the two, it looks like: assert is able to terminate properly, and I am able to see the real values going into the statement (probably it's gotten better after years of evolution, till Python 3.8). But it prints out more traceback messages, which may/may not be useful.

So I think it's ok to use assert in unittest, but it's definitely worth considering self.assertEqual as well

Why do assertions in unittest use TestCase.assertEqual not the assert keyword?

6 Answers6

Linked