21

In Python testing, why would you use assert methods:

self.assertEqual(response.status_code, 200)
self.assertIn('key', my_dict)
self.assertIsNotNone(thing)

As opposed to the direct assertions:

assert response.status_code == 200
assert 'key' in my_dict
assert thing is not None

According to the docs:

These methods are used instead of the assert statement so the test runner can accumulate all test results and produce a report

However this seems to be bogus, a test runner can accumulate results and produce a report regardless. In a related post unutbu has shown that unittest will raise an AssertionError just the same as the assert statement will, and that was over 7 years ago so it's not a shiny new feature either.

With a modern test runner such as pytest, the failure messages generated by the assertion helper methods aren't any more readable (arguably the camelCase style of unittest is less readable). So, why not just use assert statements in your tests? What are the perceived disadvantages and why haven't important projects such as CPython moved away from unittest yet?

agiledatabase
  • 29
  • 1
  • 4
wim
  • 338,267
  • 99
  • 616
  • 750
  • 2
    I work so far out of `unittest` that I use `print()` instead of `assert`. I think it all depends on the complexity and scale of your project. – TigerhawkT3 Nov 03 '15 at 05:32
  • 1
    This seems to be mostly a matter of opinion. If you think `assert` is superior then there's nothing that prevents (besides maybe your colleagues disagreeing) you from using that. However it should be known that you might override methods in the `TestCase` class and customize the behaviour - the `assert` statement doesn't have this flexibility. Also the `TestCase.failureException` may be altered which invalidates your point about it being the same exception that's thrown as from the `assert` statement (this could be used if you want to consider `assert` an error instead of failure). – skyking Nov 03 '15 at 07:10
  • Just plain `assert` is far more readable. So always use `pytest` if your tests fail. – o11c Aug 29 '17 at 20:56
  • @o11c Yeah, I agree. But the question is more about, why all these `self.assertStuff` methods exist at all and why do many well-respected libraries still actively use them. – wim Aug 29 '17 at 21:31

4 Answers4

16

The key difference between using assert keyword or dedicated methods is the output report. Note that the statement following assert is always True or False and can't contain any extra information.

assert 3 == 4

will simply show an AssertionError in the report. However,

self.assertTrue(3 == 4)

Gives some extra info: AssertionError: False is not true. Not very helpful, but consider:

self.assertEqual(3, 4)

It's much better as it tells you that AssertionError: 3 != 4. You read the report and you know what kind of assertion it was (equality test) and values involved.

Suppose you have some function, and want to assert value it returns. You can do it in two ways:

# assert statement
assert your_function_to_test() == expected_result

# unittest style
self.assertEqual(your_function_to_test(), expected_result)

In case of error, the first one gives you no information besides the assertion error, the second one tells you what is the type of assertion (equality test) and what values are involved (value returned and expected).

For small projects I never bother with unittest style as it's longer to type, but in big projects you may want to know more about the error.

warownia1
  • 2,771
  • 1
  • 22
  • 30
  • 3
    You can add an arbitrary string to an assertion, to be included in the raised `AssertionError`: `assert 3 == 4, "3 != 4"`. – chepner Aug 29 '17 at 20:17
  • 2
    More importantly, the test runner / test framework is able to automatically generate the context. – wim Aug 29 '17 at 20:34
8

I'm not entirely sure I understand the question. The title is "Why not use pythons assert statement in tests these days".

As you've noted, in fact you can use plain assertions if you use a test-framework like pytest. However pytest does something quite special to get this to work. It re-writes the plain assertions in the test-code before it runs the tests.

See https://docs.pytest.org/en/stable/writing_plugins.html#assertion-rewriting which states:

One of the main features of pytest is the use of plain assert statements and the detailed introspection of expressions upon assertion failures. This is provided by “assertion rewriting” which modifies the parsed AST before it gets compiled to bytecode.

The unittest framework does not implement this extra complexity. (And it is extra complexity. Pytest re-writes only the assertions in the test cases, it will not re-write the assertions in the other python library your test-code uses. So you will sometimes find pytest hits an assertion error in your test-code, but there's no detail about why the assertion has failed, because it hasn't re-written that bit of your code. And thus you only get a plain AssertionError without any information as to why it failed.)

Instead, unittest provides methods like assertEqual so that it can:

  1. Know it's a test assertion that has failed rather than some other unhandled/unexpected exception; and
  2. It can provide information as to why the assertion is not satisfied. (A plain assertion in python does nothing but raise AssertionError. It does not say, for example AssertionError because 1 != 2)

Pytest does number 1 and 2 by re-writing the Abstract Syntax Tree before it runs the test-code. Unittest takes the more traditional approach of asking the developer to use particular methods.

So essentially the answer is: it's an implementation difference between the test-frameworks. Put another way, Python's in-built assert statement provides no debugging information about why the failure occurred. So, if you want some more information, you need to decide how you're going to implement it.

Unittest is a lot simpler than pytest. Pytest is great but it is also a lot more complicated.

wim
  • 338,267
  • 99
  • 616
  • 750
Donal
  • 8,430
  • 3
  • 16
  • 21
  • 1
    It's better for that extra complexity to be in the test framework itself, rather than appearing in each and every test suite. The question was originally intended to be more like "_why doesn't unittest do something to introspect assertion failures, rather than ask the developer to use all these ugly helper methods_", but in hindsight it's obvious the answer is just "because they didn't implement it like that". – wim Nov 08 '21 at 17:18
3

I think with the current pytest versions you can use assert in most cases, as the context is reconstructed by the test framework (pytest 2.9.2):

    def test_requests(self):
        ...
>       self.assertEqual(cm.exception.response.status_code, 200)
E       AssertionError: 404 != 200

looks similar to

    def test_requests(self):
        ...
>       assert cm.exception.response.status_code == 200
E       AssertionError: assert 404 == 200
E         -404
E         +200

In theory, using self.assertXxx() methods would allow pytest to count the number of assertions that did not fail, but AFAIK there is no such metric.

mar10
  • 14,320
  • 5
  • 39
  • 64
1

The link to the docs that you found is the correct answer. If you do not like this style of writing tests I would highly suggest using pytest:

http://pytest.org/latest/

pytest has done a bunch of work that allows you to use the assert statement the way you want to. It also has a bunch of other really nice features such as their fixtures.

tipsqueal
  • 183
  • 5
  • 4
    I do use pytest, as mentioned in the question. How can the claim in the docs be considered correct, when the test runner happily accumulates all test results and produces a report regardless of whether you use assert statements or the helper functions? – wim Nov 03 '15 at 06:28