argparse: How to use required positionals and subparsers together?

Question

I'm encountering a simple use-case that argparse surprisingly doesn't seem to handle. I would like to have a required positional argument in addition to having multiple subparsers. The rationale is that this CLI application supports one use-case very readily with a concise syntax, with more granular options being nested under subcommands. For this post, imagine that I am writing a CLI application for tagging files with the syntax:

# tag a file
argparse_test.py <file> [<tags>...]

# list tags
argparse_test.py tags <file>

Usage example:

$ argparse_test.py ./cat.jpg cute funny
$ argparse_test.py tags ./cat.jpg
> cute, funny

My implementation is as follows:

import argparse

# initialize main parser
parser = argparse.ArgumentParser()
parser.add_argument('file', help="The file to index.")
parser.add_argument('tags', nargs='*', help="Tags to append.")

# initialize `tags` subparser
subparsers = parser.add_subparsers()
tag_subparser = subparsers.add_parser("tags")
tag_subparser.add_argument('file', help="The file to list tags for.")

# 1. Failing test case
args = parser.parse_args(["./cat.jpg", "cute", "funny"])
print(args)

# 2. Failing test case (comment out #1 to reach this)
args = parser.parse_args(["tags", "./cat.jpg"])
print(args)

Surprisingly, this fails in both cases!

For the first test case:

usage: argparse_test.py [-h] file [tags ...] {tags} ...
argparse_test.py: error: argument {tags}: invalid choice: 'funny' (choose from 'tags')

For the second test case:

usage: argparse_test.py tags [-h] file
argparse_test.py tags: error: the following arguments are required: file

Commenting out either the main parser or the subparser will fix one (but not both) test case every time. It looks like this issue stems from some conflict argparse has in dealing with required positionals and subcommands.

Is there any known way to implement support for this CLI syntax via argparse in Python 3.8+?

(BTW, I've tested this on Python 3.9, 3.10, and 3.11, all of which share the same behavior).

think of the `subparsers` as anther `positional` with `choices`. Also the string is assigned to the argument and then tested against the choices. — hpaulj, May 23 '23 at 00:16
Another thought - don't try to get the same `dest` in both the main and the sub. — hpaulj, May 23 '23 at 00:19
get rid of the `add_argument('tags', nargs='*',` argument. a variable nargs positional creats too many headaches when also using a `subparsers` positional — hpaulj, May 23 '23 at 01:31
i commented on a similar bug issue https://github.com/python/cpython/issues/103520 and on SO https://stackoverflow.com/q/75930563/901925 — hpaulj, May 23 '23 at 01:39

score 0 · Answer 1 · answered Aug 20 '23 at 02:31

I got two different solutions. The first one will work with your examples. The second one is what I recommend.

First Solution

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("action")
options, remainder = parser.parse_known_args()
if options.action == "tags":
    print(f"Show tags. Files: {remainder}")
else:
    print(f"File: {options.action}")
    print(f"Tags: {remainder}")

Basically, I would parse the "action".

If that action is "tags", then what follows (the remainder) is a list of files.
Else, that action is a file, follows by a list of tags

The problem with this approach is the interface is not consistent: in one case, the first argument is a file name; in the other case, the first argument is a command.

Second Solution

I propose another solution, which introduce two sub commands: "add-tags" and "show-tags". You can call them whatever you like.

import argparse

parser = argparse.ArgumentParser()
subparser = parser.add_subparsers(dest="action")

add_tags = subparser.add_parser("add-tags")
add_tags.add_argument("file")
add_tags.add_argument("tags", nargs="+")

show_tags = subparser.add_parser("show-tags")
show_tags.add_argument("file")

print("\n# Add tags")
args = parser.parse_args(["add-tags", "./cat.jpg", "cute", "funny"])
print(args)

print("\n# Show tags")
args = parser.parse_args(["show-tags", "./cat.jpg"])
print(args)

Output:

# Add tags
Namespace(action='add-tags', file='./cat.jpg', tags=['cute', 'funny'])

# Show tags
Namespace(action='show-tags', file='./cat.jpg')

The advantages of this approach are:

Consistent interface: the first argument is always a command
Extensibility: we can later add more sub commands such as "remove-tags", "clear-all-tags", ...

argparse: How to use required positionals and subparsers together?

1 Answers1

First Solution

Second Solution