How can I create an argparse mutually exclusive group with multiple positional parameters?

Question

I'm trying to parse command-line arguments such that the three possibilities below are possible:

script
script file1 file2 file3 …
script -p pattern

Thus, the list of files is optional. If a -p pattern option is specified, then nothing else can be on the command line. Said in a "usage" format, it would probably look like this:

script [-p pattern | file [file …]]

I thought the way to do this with Python's argparse module would be like this:

parser = argparse.ArgumentParser(prog=base)
group = parser.add_mutually_exclusive_group()
group.add_argument('-p', '--pattern', help="Operate on files that match the glob pattern")
group.add_argument('files', nargs="*", help="files to operate on")
args = parser.parse_args()

But Python complains that my positional argument needs to be optional:

Traceback (most recent call last):
  File "script", line 92, in <module>
    group.add_argument('files', nargs="*", help="files to operate on")
…
ValueError: mutually exclusive arguments must be optional

But the argparse documentation says that the "*" argument to nargs meant that it is optional.

I haven't been able to find any other value for nargs that does the trick either. The closest I've come is using nargs="?", but that only grabs one file, not an optional list of any number.

Is it possible to compose this kind of argument syntax using argparse?

Might there be a way to do this with "subparsers"? https://docs.python.org/2/library/argparse.html#sub-commands — mac, Mar 21 '18 at 18:27
Not a bad idea, @mac. Subparsers do seem to operate independently of one another. But my quick reading of the documentation suggests to me that the choice of which subparser is invoked is done by an earlier positional argument. (They use the example of parameters to the `svn` command.) I don't have any such selection mechanism I could use here in the desired command line arguments. — seanahern, Mar 22 '18 at 18:32
yeah, this seems to be a major limitation; too bad there isn't support for a "default" subparser that would be used when the args don't contain any of the other named subparsers — mac, Mar 23 '18 at 13:26

hpaulj · Accepted Answer · 2016-08-10T01:24:49.720

short answer

Add a default to the * positional

long

The code that is raising the error is,

    if action.required:
        msg = _('mutually exclusive arguments must be optional')
        raise ValueError(msg)

If I add a * to the parser, I see that the required attribute is set:

In [396]: a=p.add_argument('bar',nargs='*')
In [397]: a
Out[397]: _StoreAction(option_strings=[], dest='bar', nargs='*', const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [398]: a.required
Out[398]: True

while for a ? it would be False. I'll have dig a bit further in the code to see why the difference. It could be a bug or overlooked 'feature', or there might a good reason. A tricky thing with 'optional' positionals is that no-answer is an answer, that is, an empty list of values is valid.

In [399]: args=p.parse_args([])
In [400]: args
Out[400]: Namespace(bar=[], ....)

So the mutually_exclusive has to have some way to distinguish between a default [] and real [].

For now I'd suggest using --files, a flagged argument rather than a positional one if you expect argparse to perform the mutually exclusive testing.

The code that sets the required attribute of a positional is:

    # mark positional arguments as required if at least one is
    # always required
    if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]:
        kwargs['required'] = True
    if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs:
        kwargs['required'] = True

So the solution is to specify a default for the *

In [401]: p=argparse.ArgumentParser()
In [402]: g=p.add_mutually_exclusive_group()
In [403]: g.add_argument('--foo')
Out[403]: _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [404]: g.add_argument('files',nargs='*',default=None)
Out[404]: _StoreAction(option_strings=[], dest='files', nargs='*', const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [405]: p.parse_args([])
Out[405]: Namespace(files=[], foo=None)

The default could even be []. The parser is able to distinguish between the default you provide and the one it uses if none is given.

oops - default=None was wrong. It passes the add_argument and required test, but produces the mutually_exclusive error. Details lie in how the code distinguishes between user defined defaults and the automatic ones. So use anything but None.

I don't see anything in the documentation about this. I'll have to check the bug/issues to see it the topic has been discussed. It's probably come up on SO before as well.

NIce, but even adding `default=None` to the `files` positional argument, I still get an error if I run `p.parse_args(["-p", "foo"])`: `error: argument files: not allowed with argument -p/--pattern` — seanahern, Jan 27 '16 at 18:59
Yes, `default=[]` seems to be helping. Lemme do a bit more testing. This may have done the trick. — seanahern, Jan 27 '16 at 19:03
That did the trick entirely. Adding `default=[]` to the argument with the `nargs="*"` gives the right result in all cases. — seanahern, Jan 27 '16 at 19:12
With `None`, you are getting the mutually_exclusive error. It's result of how it tests for user given defaults v automatic ones. So anything but `None` is needed. — hpaulj, Jan 27 '16 at 19:16

score 0 · Answer 2 · answered Jan 27 '16 at 18:31

You are trying to use the 'files' argument to catch a number of files but you are not supplying it on the cmdline examples. I think the library gets confused that you're not using the dash prefix. I would suggest the following:

import argparse

parser = argparse.ArgumentParser(prog="base")
group = parser.add_mutually_exclusive_group()
group.add_argument('-p', '--pattern', action="store", help="Operate on files that match the glob pattern")
group.add_argument('-f', '--files',  nargs="*", action="store", help="files to operate on")

args = parser.parse_args()
print args.pattern
print args.files

I don't want to use a dash prefix. I want them all to be positional arguments. — seanahern, Jan 27 '16 at 18:43

score -1 · Answer 3 · answered Jan 27 '16 at 18:35

-1

    import argparse
    parse  = argparse.ArgumentParser()
    parse.add_argument("-p",'--pattern',help="Operates on File")
    parse.add_argument("files",nargs = "*",help="Files to operate on")

    arglist = parse.parse_args(["-p","pattern"])
    print arglist
    arglist = parse.parse_args()
    print arglist
    arglist = parse.parse_args(["file1","file2","file3"])
    print arglist

answered Jan 27 '16 at 18:35

Arijit

175
1
5

Yes, but `arglist = parse.parse_args(["-p", "pattern", "file1","file2","file3"])` should give an error, and it does not. – seanahern Jan 27 '16 at 18:46
Namespace(files=['file1', 'file2', 'file3'], pattern='pattern') It wont give any error as optional argument "-p" by default takes one argument so (pattern ='pattern') and rest arguments will assigned to positional argument (files = ['file1','file2','file3']) – Arijit Jan 27 '16 at 19:14
Exactly. Because you skipped the mutually exclusive group, argparse is not throwing an error for `-p pat file1 file2 file3` as it should. – seanahern Jan 27 '16 at 19:15

How can I create an argparse mutually exclusive group with multiple positional parameters?

3 Answers3

short answer

long

Linked