Configure argparse to accept quoted arguments

Question

I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.

Here is a basic layout of the arguments that I am using

parser.add_argument('-M', '--module',
                    help="Module to run on changed files - should be in format MODULE:CLASS\n\
                          Specified class must have function with the signature run(src, dest)\
                          and return 0 upon success",
                    required=True)
parser.add_argument('-A', '--module_args',
                    help="Arg to be passed through to the specified module",
                    action='append',
                    default=[])

However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument

Any ideas?

Answer 1

The solution is to accept arbitrary arguments - there's an example in argparse's doc here:

argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:

>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')

Answer 2

With:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
                    help="Module to run on changed files - should be in format MODULE:CLASS\n\
                          Specified class must have function with the signature run(src, dest)\
                          and return 0 upon success",
                    )
parser.add_argument('-A', '--module_args',
                    help="Arg to be passed through to the specified module",
                    action='append',
                    default=[])
import sys
print(sys.argv)
print(parser.parse_args())

I get:

1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])

This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.

Without the quotes the -f is separate and interpreted as a flag.

1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument

Are you using windows or some other OS/shell that doesn't handle quotes the same way?

---

In Argparse `append` not working as expected

you asked about a slightly different command line:

1032:~/mypy$ python stack45146728.py  -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument

As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.

There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.

Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:

    # if it contains a space, it was meant to be a positional
    if ' ' in arg_string:
        return None

http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.