2

Similar to this question, except desiring to exit only on errors encountered, not on failures.

In theory, this would be useful because:

  • A "failed" test indicates something wrong with that test, however there should be no impact to other tests, and so the test suite continuing to execute would be valid.
  • An "error" test indicates something is wrong with the testbed/setup/config, so the test suite continuing to run would be a waste of time.

I surmise there would be elaborate ways to get this accomplished with decorators and pytest.mark, but looking for a clean, simple solution that would avoid that if possible.

Jon
  • 952
  • 1
  • 11
  • 17
  • Are you saying that failing tests because are not a problem? Why you test then in the first place if your tests are not reliable and cannot be trusted? Fix your tests instead of wasting the effort to increase technical debt of your project. – Marcin Orlowski Apr 24 '23 at 18:51
  • @MarcinOrlowski That's not what's being asked. By default, pytest runs all tests to completion, even if failures or errors are encountered. The linked question shows how to immediately stop pytest on the first failure OR error encountered. I would like to exit only on the first error encountered, not on the first failure (nor any number of failures). – Jon Apr 24 '23 at 19:11
  • That's why that's not an answer but a comment. I also understand your needs, yet I have hard times seeing any benefits from your approach. If something is broken then fix it and if you can't just clearly mark it skipped. Isn't that cleaner? – Marcin Orlowski Apr 24 '23 at 19:13
  • 1
    I still don't think you understand what's being asked for here. Either way, we are marking the tests as "failing" and they will be flagged in order to be fixed. What is being asked is _how soon we stop the testing suite_ when encountering a failure. By default, pytest keeps running all tests until all have attempted to be run, even if failures have already occurred. The `-x` option allows immediate exit upon the first failure OR error seen, but I want the exit to only occur on the first error seen. – Jon Apr 26 '23 at 05:52

0 Answers0