Argparse `append` not working as expected

Question

I am trying to configure argparse to allow me to specify arguments that will be passed onto another module down the road. My desired functionality would allow me to insert arguments such as -A "-f filepath" -A "-t" and produce a list such as ['-f filepath', '-t'].

In the docs it seems that adding action='append' should do exactly this - however I am getting an error when attempting to specify the -A argument more than once.

Here is my argument entry:

parser.add_argument('-A', '--module-args',
                    help="Arg to be passed through to the specified module",
                    action='append')

Running python my_program.py -A "-k filepath" -A "-t" produces this error from argparse:

my_program.py: error: argument -A/--module-args: expected one argument

Minimal example:

from mdconf import ArgumentParser
import sys

def parse_args():
    parser = ArgumentParser()
    parser.add_argument('-A', '--module-args',
                        help="Arg to be passed through to the module",
                        action='append')
    return parser.parse_args()

def main(args=None):
    try:
        args = parse_args()
    except Exception as ex:
        print("Exception: {}".format(ex))
        return 1
    print(args)
    return 0

if __name__ == "__main__":
    sys.exit(main())

Any ideas? I find it strange that it is telling me that it expects one argument when the append should be putting these things into a list.

Ah. It's not that you're not allowed to pass more than one `-A`. It's that it's parsing `-t` as another option, **not** an argument to `-A`, and thus the second `-A` as being passed zero arguments. — Charles Duffy, Jul 17 '17 at 17:25
BTW, `-A -t` and `-A "-t"` are **exactly** the same thing. Python has no way of knowing which one you passed, since the syntactical quotes are parsed and removed by the shell before it's ever invoked. — Charles Duffy, Jul 17 '17 at 17:28
Alternatively to those answers, look into the nargs argument. If you set nargs parameter = to `+` then you can just accept more than one argument and you don't need this append command. — JoshKopen, Jul 17 '17 at 17:29
@JoshKopen, I don't believe that actually helps here. You still need it to treat the `-t` as an extra argument to `-A`, as opposed to a new, separate, invalid argument on its own. — Charles Duffy, Jul 17 '17 at 17:37
To see rationale behind this behavior, btw, see the documentation quoted in https://stackoverflow.com/a/14693762/14122 — Charles Duffy, Jul 17 '17 at 17:45
@CharlesDuffy fair point, I think my solution would honestly be easier for the user if you added those steps but he would have to change how he is doing it fundamentally. More of a suggestion than anything else to move forward (why I kept it in the comments). — JoshKopen, Jul 17 '17 at 17:49
@JoshKopen, whether that's an acceptable solution at all depends on details we aren't privy to. Perhaps the user already *is* taking a list of input files as their positional arguments, so they can't also take module parameters that way. I've used the same approach the OP is taking here in the past, and when I used it, it was because I *couldn't* go the other route. — Charles Duffy, Jul 17 '17 at 17:50
@JoshKopen as @CharlesDuffy mentioned `nargs='*'` gives the same issues (tries to parse `-t` as its own arg) so the leading space gives the same workaround. — wakey, Jul 17 '17 at 18:00
I touched on this in my answer to your previous question, https://stackoverflow.com/a/45150332/901925. The `-t` is interpreted as a flag, the `-f xxx` is not. — hpaulj, Jul 17 '17 at 18:02
@wKavey you would have to do a much bigger work around, the accepted answer is definitely better for the time being. — JoshKopen, Jul 17 '17 at 18:02
It will be simpler if you avoid arguments that don't start with the flag '-'. That kind of nesting confuses both `argparse`, and potentially your users as well. — hpaulj, Jul 17 '17 at 18:11
@hpaulj, ...somewhat necessary if one is telling a program how to invoke a second program, however. — Charles Duffy, Jul 17 '17 at 19:32

Charles Duffy · Accepted Answer · 2017-07-17T17:30:40.443

0

The problem isn't that -A isn't allowed to be called more than once. It's that the -t is seen as a separate option, not an argument to the -A option.

As a crude workaround, you can prefix a space:

python my_program.py \
  -A " -k filepath" \
  -A " -t"

Given the following Minimal, Complete and Verifiable Example:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-A', '--module-args',
                    help="Arg to be passed through to the specified module",
                    action='append')
args = parser.parse_args()
print repr(args.module_args)

...that usage returns:

[' -k filepath', ' -t']

whereas leaving off the leading spaces reproduces your error.

edited Jul 17 '17 at 17:30

answered Jul 17 '17 at 17:27

Charles Duffy

280,126
43
390
441

This sounds like the reason - however it is able to parse the `-A "-k filepath"` correctly with no spacing added. Additionally, adding the space is giving me a `my_program.py: error: unrecognized arguments: -t` – wakey Jul 17 '17 at 17:30
Yes, I noticed that too; would need to look into the details of the argparse code to tell you why, but absent that, I'm assuming it's just running a simple regex that looks for something that starts with a dash *and doesn't include a space*. – Charles Duffy Jul 17 '17 at 17:31
1

@wKavey, yes, I assume you'll do the work in your program of stripping those leading spaces off later. (Thus, `actualModuleArgs = [ s.lstrip(' ') for s in args.module_args ]` or such). That's unfortunate, but... well, it's why I branded this a *crude workaround*. – Charles Duffy Jul 17 '17 at 17:32
@wKavey, that said, `-k filepath` being one argument led me to believe that you were expecting string-splitting to happen anyhow. Otherwise it would be `-A -k -A filepath`. – Charles Duffy Jul 17 '17 at 17:34
1

See [my answer here](https://stackoverflow.com/a/16175115/1399279): Use an `=`, as in `-A=-t` – SethMMorton Jul 19 '17 at 02:57
@SethMMorton, excellent, thanks – Charles Duffy Jul 19 '17 at 13:14

Argparse `append` not working as expected

1 Answers1

Linked