9

I'm using argparse to parse the inputs to my python3 program. I was recently asked to range check some of the numeric inputs, a seemingly good idea. Argparse has a facility to do just that.

The numeric inputs are port numbers, in the usual range 0-65535, so I altered my parse command line to :

import argparse
cmd_parser = argparse.ArgumentParser()
cmd_parser = add_argument('-p', help='Port number to connect to', dest='cmd_port', default=1234, type=int, choices=range(0,65536))
cmd_parser.parse_args(['-h'])

Now, however, when I request the help, I get flooded with all the possible values from argparse. eg.

optional arguments:
    -h, --help            show this help message and exit
    -p {0,1,2,3,4,5,6,7,8,9,10,11,12,13 ...
    65478,65479,65480,65481,65482,65483,65484,65485,65486,65487,65488,65489,
    65490,65491,65492,65493,65494,65495,65496,65497,65498,65499,65500,65501,
    65502,65503,65504,65505,65506,65507,65508,65509,65510,65511,65512,65513,
    65514,65515,65516,65517,65518,65519,65520,65521,65522,65523,65524,65525,
    65526,65527,65528,65529,65530,65531,65532,65533,65534,65535}
                    Port number to connect to
...

It lists every single port in that range. Is there a way to truncate this or make it realize its a range (0-65535) or for it to use ellipsis or something to make it a bit prettier? Is my only option to explicitly range check my inputs with if statements?

I've been googling this but I'm having trouble finding examples where people used argparse and specified choices. I also checked the documentation on argparse but didn't see anything useful. https://docs.python.org/2/library/argparse.html

LawfulEvil
  • 2,267
  • 24
  • 46
  • did you try monkey patching `parser.print_help()` to get the required output ? – cmidi Jun 07 '16 at 13:36
  • The print_help() command shows the same exaggerated output (every number between 0 and 65535 is shown). – LawfulEvil Jun 07 '16 at 13:38
  • I'd try to open a bug in python's issue tracker. When generating the output it seems like `argparse` is iterating over all possible choices and displaying all of them. They could add an option to only show up to `N` choices if they are available (like first `N-1` and the last one). – Bakuriu Jun 07 '16 at 13:42
  • BTW: pretty closely related question: http://stackoverflow.com/questions/9366369/python-argparse-lots-of-choices-results-in-ugly-help-output I'm not closing as a duplicate since I don't find those answers satisfying and years have passed. – Bakuriu Jun 07 '16 at 13:44
  • I see I mentioned the bug/issue in that SO link as well. Anyways, with your numeric range I think it's easier to test with a custom `type` rather than patch the `choices` listings. – hpaulj Jun 07 '16 at 16:15

6 Answers6

6

Use custom action...

import argparse

class PortAction(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        if not 0 < values < 2**16:
            raise argparse.ArgumentError(self, "port numbers must be between 0 and 2**16")
        setattr(namespace, self.dest, values)

cmd_parser = argparse.ArgumentParser()
cmd_parser.add_argument('-p',
                        help='Port number to connect to',
                        dest='cmd_port',
                        default=1234,
                        type=int,
                        action=PortAction,
                        metavar="{0..65535}")

An invalid port number will display the error message based on the raised ArgumentError. If you enter a value of 65536, the following line will be printed:

error: argument -p: port numbers must be between 0 and 2**16

The usage and help messages will be printed based on the metavar displayed

arewm
  • 649
  • 3
  • 12
2

Just use int as type in add_argument, and manually verify it's in the permitted range. Or, use a type of your own, which has a constructor that does the checking for you, and a __int__ method for implicit conversion:

class portnumber:
    def __init__(self, string):
        self._val = int(string)
        if (not self._val > 0) or (not self.val < 2**16):
            raise argparse.ArgumentTypeError("port numbers must be integers between 0 and 2**16")
    def __int__(self):
        return self._val

...

parser.add_argument("-p",type=portnumber)
Marcus Müller
  • 34,677
  • 4
  • 53
  • 94
2

Supply an explicit metavar argument instead of letting argparse produce one for you.

cmd_parser.add_argument('-p',
                        help='Port number to connect to',
                        dest='cmd_port',
                        default=1234,
                        type=int,
                        choices=range(0,65536),
                        metavar="{0..65535}")
chepner
  • 497,756
  • 71
  • 530
  • 681
  • 1
    This doesn't quite do it. For alphanumeric inputs, it is fine. For the help, it makes it look nicer, but if I provide a parameter out of range it still displays the full list of options. eg. -p -5, then it shows all the numbers. I think I'll have to combine this with the port number class to achieve the best results. – LawfulEvil Jun 07 '16 at 13:59
  • 1
    Oh, good grief. I'm converted; this should be fixed in `argparse` itself :) – chepner Jun 07 '16 at 14:00
  • @LawfulEvil, see my answer to solve the parameter out of range issue. It is similar to the port number class, but I feel like it is more argparsey... – arewm Jun 07 '16 at 15:01
2

There is a Python bug/issue regarding the formatting of large choices lists. Currently choices in the help is formatted with

    def _metavar_formatter: ...
        choice_strs = [str(choice) for choice in action.choices]
        result = '{%s}' % ','.join(choice_strs)

and for errors with:

def _check_value(self, action, value):
    # converted value must be one of the choices (if specified)
    if action.choices is not None and value not in action.choices:
        args = {'value': value,
                'choices': ', '.join(map(repr, action.choices))}
        msg = _('invalid choice: %(value)r (choose from %(choices)s)')
        raise ArgumentError(action, msg % args)

So it expects choices to be an iterable, but does nothing to compress the list or make it pretty. Note that the only test is that value not in action.choices. choices is a very simple feature.

I mentioned this issue in a previous SO question: http://bugs.python.org/issue16468. Proposed patches are involved, so don't expect any fix soon.

I would recommend using your own type testing rather than choices. Or do your own range testing after parsing.

def myrange(astring):
   anint = int(astring)
   if anint in range(0,1000):
        return anint
   else:
        raise ValueError()
        # or for a custom error message
        # raise argparse.ArgumentTypeError('valid range is ...')

parser.add_argument('value',type=myrange,metavar='INT',help='...')

Another old (2012) SO question that addresses the range choices. The answer suggests both a help-formatter fix and the custom type

Python's argparse choices constrained printing

and

Python argparse choices from an infinite set

============================

Out of curiosity I defined a custom range class. It behaves like a regular range (without the step) for the in test, but returns custom values when used as an iterator.

class Range(object):
    def __init__(self, start, stop, n=3):
        self.start = start
        self.stop = stop
        self.n = n
    def __contains__(self, key):
        return self.start<=key<self.stop   
    def __iter__(self):
        if self.stop<(self.start+(self.n*3)):
            for i in range(self.start, self.stop):
                yield i
        else:
            for i in range(self.start, self.start+self.n):
                yield i
            yield '...'
            for i in range(self.stop-self.n, self.stop):
                yield i

When used with an argument as

parser.add_argument("-p",type=int, choices=Range(2,10,2))

It produces

1455:~/mypy$ python stack37680645.py -p 3
Namespace(p=3)

1458:~/mypy$ python stack37680645.py -h
usage: stack37680645.py [-h] [-p {2,3,...,8,9}]

optional arguments:
  -h, --help        show this help message and exit
  -p {2,3,...,8,9}

The error message isn't quite what I'd want, but close

1458:~/mypy$ python stack37680645.py -p 30
usage: stack37680645.py [-h] [-p {2,3,...,8,9}]
stack37680645.py: error: argument -p: invalid choice: 30 (choose from 2, 3, '...', 8, 9)

It would be better if the choices formatting allowed the action.choices object to create its own str or repr string.

Actually the iter could be as simple as (just one string):

def __iter__(self):
    yield 'a custom list'

Another option is to use metavar to control the usage/help display, and this __iter__ to control the error display.

One thing to watch out for when using metavar. The usage formatter does not behave when there are special characters like space, '()' and '[]' in the metavar, especially when the usage line extends to 2 or more lines. That's a known bug/issue.

Community
  • 1
  • 1
hpaulj
  • 221,503
  • 14
  • 230
  • 353
0

Monkey patch the print_help to get the desired output

def my_help(): print "0-65535 range"
cmd_parser = argparse.ArgumentParser()
cmd_parser.add_argument('-p', help='Port number to connect to',   dest='cmd_port', default=1234, type=int, choices=range(0,65536))
cmd_parser.print_help = my_help
cmd_parser.parse_args()
cmidi
  • 1,880
  • 3
  • 20
  • 35
0

The answer I used is inspired by @hpaulj's answer. However, I agree that the problem really lies at the feet of argparse. I found this SO query when my computer hung trying to allocate gigabytes of space just to output the help text.

My issue with hpaulj's 'Range' class is that when using large upper limits, the message still pushes out very large numbers. The following class uses a infinite upper limit by default.

class ArgRange(object):
    from decimal import Decimal
    huge = Decimal('+infinity')
    huge_str = '{:.4E}'.format(huge)

    def __init__(self, start, stop=huge, n=3):
        self.start = start
        self.stop = stop
        self.n = n

    def __contains__(self, key):
        return self.start <= key < self.stop

    def __iter__(self):
        if self.stop < self.start+(self.n*3):
            for i in range(self.start, self.stop):
                yield i
        else:
            for i in range(self.start, self.start+self.n):
                yield I
            if self.stop is self.huge:
                yield '...' + huge_str
            else:
                yield '...'
                for i in range(self.stop - self.n, self.stop):
                    yield i

bounds = ArgRange(2)
balance = ArgRange(0, 1000)
parser = argparse.ArgumentParser(description="Do something fun")
parser.add_argument("width", type=int, choices=bounds,  default=9)
parser.add_argument("height", type=int, choices=balance, default=200)

With an incorrect value, the error is:

argument width: invalid choice: 1 (choose from 2, 3, 4, '...infinity')

or

argument height: invalid choice: 2000 (choose from 0, 1, 2, '...', 997, 998, 999)

And usage looks like:

usage: test.py 
           {2,3,4,...infinity} {0,1,2,...,997,998,999}
Konchog
  • 1,920
  • 19
  • 23