-3

We are using Django with unittest. Some tests are skipped with the @unittest.skip decorator. But if I run the tests with Python 3.6 or 3.7, I get a number of tests passed (Ran 993 tests / OK), and if I run the same tests with Python 3.8, I get the same number of tests but with some tests skipped (Ran 993 tests / OK (skipped=4)). I would like to know if the same tests were also skipped with Python 3.6 and 3.7, or only with Python 3.8? And why do I get the skipped output only with Python 3.8? And is the number 993 including the skipped tests or not including them? Is one of the outputs incorrect? Because it doesn't make sense that the output is different for different versions of Python and I don't understand the reason for this difference. I didn't find it documented in the documentation.

Our code is open source, and you can see for example a skipped test here.

Update: I added 4 more tests with the decorator @unittest.skip. When I run all the tests with Python 3.8, I get this output: Ran 997 tests / OK (skipped=8) (with an s for every skipped test, like before). But if I run the tests with Python 3.6 or 3.7, I get this output: Ran 997 tests / OK and there are no s in the output, like before (I get 997 dots). Although I have 4 more tests than before.

The tests I added raise an exception, so if they would not be skipped they would fail.

I think the skipped tests are skipped in all Python versions, but in Python 3.6 and 3.7 there is no output about them being skipped. Is it a bug?

Uri
  • 2,992
  • 8
  • 43
  • 86
  • 1
    This seems ludicrously easy to test. Simply add a new skipped test and see if "Ran 993 tests" becomes "Ran 994 tests" on 3.7 – Adam Smith Feb 11 '21 at 19:23
  • @AdamSmith I'll check and let you know. This can take some time since the tests are slow. – Uri Feb 11 '21 at 19:26
  • @MrBeanBremen Please see my update. Can you explain this? – Uri Feb 11 '21 at 19:47
  • @MrBeanBremen The number 993 is the total number of tests, including skipped tests. – Uri Feb 11 '21 at 20:11

1 Answers1

2

Yes, it is a bug, but in your code. You are using @unittest.skip instead of @unittest.skip(reason). Figuring out if and how exactly the implementation of skip() changed between 3.7 and 3.8 to cause a change in behavior, or if it is something entirely different, is left as an exercise.

When skip() is used correctly, you get ss printed and the skipped tests reported with Python 3.7.

Shai Berger
  • 2,963
  • 1
  • 20
  • 14
  • Thank you! I was not aware that `reason` is required and that the test is not skipped without it. – Uri Feb 12 '21 at 04:00