9

I want to use pytest to check if the argparse.ArgumentTypeError exception is raised for an incorrect argument:

import argparse
import os
import pytest


def main(argsIn):

    def configFile_validation(configFile):
        if not os.path.exists(configFile):
            msg = 'Configuration file "{}" not found!'.format(configFile)
            raise argparse.ArgumentTypeError(msg)
        return configFile

    parser = argparse.ArgumentParser()
    parser.add_argument('-c', '--configFile', help='Path to configuration file', dest='configFile', required=True, type=configFile_validation)
    args = parser.parse_args(argsIn)


def test_non_existing_config_file():
    with pytest.raises(argparse.ArgumentTypeError):
        main(['--configFile', 'non_existing_config_file.json'])

However, running pytest says During handling of the above exception, another exception occurred: and consequently the test fails. What am I doing wrong?

Drise
  • 4,310
  • 5
  • 41
  • 66
pipppoo
  • 133
  • 2
  • 8
  • The `parser._get_value` method catches the `ArgumentTypeError` and raises a `ArgumentError`. That is turn is caught in `parse_known_args` which exits via `parser.error`. System unittest file, `test_argparse.py` subclasses `ArgumentParser` to redefine the `error` method. – hpaulj Mar 16 '18 at 17:25
  • 1
    Note that `_get_value` also catches `TypeError` and `ValueError`. The difference is that the `ArgumentTypeError` `msg` is passed on up to the error message, while the others use a standardized message. So a full test would need to check the message content as well as the `exit`. – hpaulj Mar 16 '18 at 18:18
  • Related question (for `unittest` instead of `pytest`, but similar approach I guess): [How can I test whether my code is throwing the appropriate argparse exceptions?](https://stackoverflow.com/q/40898755/1804173) – bluenote10 Sep 27 '22 at 22:15

4 Answers4

7

The problem is that if argument's type converter raises exception ArgumentTypeError agrparse exits with error code 2, and exiting means raising builtin exception SystemExit. So you have to catch that exception and verify that the original exception is of a proper type:

def test_non_existing_config_file():
    try:
        main(['--configFile', 'non_existing_config_file.json'])
    except SystemExit as e:
        assert isinstance(e.__context__, argparse.ArgumentError)
    else:
        raise ValueError("Exception not raised")
phd
  • 82,685
  • 13
  • 120
  • 165
  • Does the 2nd `raises` test do anything in this context? – hpaulj Mar 16 '18 at 18:19
  • @hpaulj Good catch! The case is complex. The code have to catch `SystemExit` and verify that the original exception is of `ArgumentTypeError`. I updated the answer. And found that the real exception is of `ArgumentError`. – phd Mar 16 '18 at 19:00
  • 1
    Just to add, if you want to verify an error message such as if you provide a wrong choice to an argument that specified choices, you can check against `err.__context__.message` – Dan Jun 02 '20 at 14:29
1

Here's the ArgumentTypeError test in the test_argparse.py file (found in the development repository)

ErrorRaisingAgumentParser is a subclass defined at the start of the file, which redefines the parser.error method, so it doesn't exit, and puts the error message on stderr. That part's a bit complicated.

Because of the redirection I described the comment, it can't directly test for ArgumentTypeError. Instead it has to test for its message.

# =======================
# ArgumentTypeError tests
# =======================

class TestArgumentTypeError(TestCase):

    def test_argument_type_error(self):

        def spam(string):
            raise argparse.ArgumentTypeError('spam!')

        parser = ErrorRaisingArgumentParser(prog='PROG', add_help=False)
        parser.add_argument('x', type=spam)
        with self.assertRaises(ArgumentParserError) as cm:
            parser.parse_args(['XXX'])
        self.assertEqual('usage: PROG x\nPROG: error: argument x: spam!\n',
                         cm.exception.stderr)
hpaulj
  • 221,503
  • 14
  • 230
  • 353
1

Using pytest you can do the following in order to check that argparse.ArugmentError is raised. Additionally, you can check the error message.

with pytest.raises(SystemExit) as e:
    main(['--configFile', 'non_existing_config_file.json'])

assert isinstance(e.value.__context__, argparse.ArgumentError)
assert 'expected err msg' in e.value.__context__.message
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
0

Inspired by @Giorgos's answer, here is a small context manager that makes the message extraction a bit more re-usable. I'm defining the following in a common place:

import argparse
import pytest
from typing import Generator, Optional


class ArgparseErrorWrapper:
    def __init__(self):
        self._error: Optional[argparse.ArgumentError] = None

    @property
    def error(self):
        assert self._error is not None
        return self._error

    @error.setter
    def error(self, value: object):
        assert isinstance(value, argparse.ArgumentError)
        self._error = value


@contextmanager
def argparse_error() -> Generator[ArgparseErrorWrapper, None, None]:
    wrapper = ArgparseErrorWrapper()

    with pytest.raises(SystemExit) as e:
        yield wrapper

    wrapper.error = e.value.__context__

This allows to test for parser errors concisely:

def test_something():
    with argparse_error() as e:
        # some parse_args call here
        ...
    assert "Expected error message" == str(e.error)

bluenote10
  • 23,414
  • 14
  • 122
  • 178