1

Let's say that, for whatever reason, I want to write a set of test cases that start out from the premise of failing code.

Maybe because setting up a failure is really complicated, while the demonstrating a fixed state is simple. In my case, it's not even that, I wanted to start with a failed test and then show how to fix it, for documentation purposes.

I can decorate @unittest.expectedFailure on the base class.

But the the fixed subclasses blow up with unexpected success because the decoration is inherited.

Can I remove the expectedFailure somehow?

  • In the code itself, not in a command line argument?

  • While I use and appreciate pytest this is a question for regular unittest.

  • unittest.skipXXX is not what I want, I do want to run Test_ThisShouldFail's test.

import sys
import unittest

@unittest.expectedFailure
class Test_ThisShouldFail(unittest.TestCase):
    """ for argument's sake, let's say 
    the configuration and testing is very complicated 
    and I want it fail by default.
    Subclasses fix the issue but re-use the test code
    """

    data = dict(a=1, b=2)

    def test_evens(self):
        for key, v in self.data.items():
            self.assertFalse(v % 2, f"{key}:Odd on {v}")

#@ ???unittest.expectedSuccess???? 
class Test_ThisShouldWork(Test_ThisShouldFail):
    """ how do I turn off the expected failure? """

    def setUp(self):
        self.data.update(a=10)

if __name__ == "__main__":
    sys.exit(unittest.main())

output:

FAILED (expected failures=1, unexpected successes=1)
(venv) @explore$ py test_expectedfail.py -v
test_evens (__main__.Test_ThisShouldFail) ... expected failure
test_evens (__main__.Test_ThisShouldWork) ... unexpected success

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (expected failures=1, unexpected successes=1)

this didn't work:

I was hoping the MRO would look at TurnItOff's blank unittest settings and use them. No such luck.

class TurnItOff(unittest.TestCase):
    pass

class Test_ThisShouldWork(TurnItOff, Test_ThisShouldFail):
    ....
JL Peyret
  • 10,917
  • 2
  • 54
  • 73

1 Answers1

1

This relies on the internal implementation of unittest.expectedFailure, but works for a start:

def expectedSuccess(test_item):
    test_item.__unittest_expecting_failure__ = False
    return test_item


@unittest.expectedFailure
class TestFailure(unittest.TestCase):
    def test_something(self):
        self.assertTrue(False)


@expectedSuccess
class TestSuccess(TestFailure):
    def test_something(self):
        self.assertTrue(True)

Note that test_item can be both a class or a function, depending on where you put the decorator.

MrBean Bremen
  • 14,916
  • 3
  • 26
  • 46
  • Clever, although a ‘delattr‘ might be even better - just in case the test system switches to using ‘hasattr‘ at some point without testing value. Lemme try it. – JL Peyret Feb 27 '20 at 17:41
  • True, though I don't think they would change that - usually this is the kind of internals other tools would depend on. – MrBean Bremen Feb 27 '20 at 18:31
  • You're probably right and it works just fine in any case. Txs. – JL Peyret Feb 28 '20 at 02:14