Is there any way to define a Python function with leading optional arguments?

Question

As we know, optional arguments must be at the end of the arguments list, like below:

def func(arg1, arg2, ..., argN=default)

I saw some exceptions in the PyTorch package. For example, we can find this issue in torch.randint. As it is shown, it has a leading optional argument in its positional arguments! How could be possible?

Docstring:
randint(low=0, high, size, \*, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor

How can we define a function in a similar way as above?

Answer 1

My other answer was about reverse-engineering the torch library, however I want to dedicate this answer on how a similar mechanism can be achieved in a non-hacky, straight forward way.

We have the multipledispatch library:

A relatively sane approach to multiple dispatch in Python. This implementation of multiple dispatch is efficient, mostly complete, performs static analysis to avoid conflicts, and provides optional namespace support. It looks good too.

So let's utilize it:

from multipledispatch import dispatch

@dispatch(int, int)
def randint(low, high):
    my_randint(low, high)

@dispatch(int)
def randint(high):
    my_randint(0, high)

def my_randint(low, high):
    print(low, high)

# 0 5
randint(5)

# 2 3
randint(2, 3)

Answer 2

Your discovery fascinated me, as it's indeed illegal in Python (and all other languages I know) to have leading optional arguments, that would surely raise in our case:

SyntaxError: non-default argument follows default argument

I got suspicious, yet I've searched on the source code:

I found, at lines 566-596 of TensorFactories.cpp that there are actually several (!) implementations of randint:

// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ randint ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Tensor randint(int64_t high, IntArrayRef size, const TensorOptions& options) {
  return native::randint(high, size, c10::nullopt, options);
}

Tensor randint(
    int64_t high,
    IntArrayRef size,
    c10::optional<Generator> generator,
    const TensorOptions& options) {
  return native::randint(0, high, size, generator, options);
}

Tensor randint(
    int64_t low,
    int64_t high,
    IntArrayRef size,
    const TensorOptions& options) {
  return native::randint(low, high, size, c10::nullopt, options);
}

Tensor randint(
    int64_t low,
    int64_t high,
    IntArrayRef size,
    c10::optional<Generator> generator,
    const TensorOptions& options) {
  auto result = at::empty(size, options);
  return result.random_(low, high, generator);
}

This pattern reoccurred at lines 466-471 of gen_pyi.py where it generates type signatures for top-level functions:

        'randint': ['def randint(low: _int, high: _int, size: _size, *,'
                    ' generator: Optional[Generator]=None, {}) -> Tensor: ...'
                    .format(FACTORY_PARAMS),
                    'def randint(high: _int, size: _size, *,'
                    ' generator: Optional[Generator]=None, {}) -> Tensor: ...'
                    .format(FACTORY_PARAMS)],

So, what basically happens is that there is no "real" optional parameter rather than several functions, in which one is present and in the other, it's not.

That means, when randint is called without the low parameter it is set as 0:

Tensor randint(
    int64_t high,
    IntArrayRef size,
    c10::optional<Generator> generator,
    const TensorOptions& options) {
  return native::randint(0, high, size, generator, options);
}

Further research, as for OP request on how that possible that there are multiple functions with the same name and different arguments:

Returning once again to gen_pyi.py we see that these functions are collected to unsorted_function_hints defined at line 436, then it's used to create function_hints at lines 509-513, and finally function_hints is set to env at line 670.

The env dictionary is used to write pyi stub files.

These stub files make use of Function/method overloading as described in PEP-484.

Function/method overloading, make use of @overload decorator:

The @overload decorator allows describing functions and methods that support multiple different combinations of argument types. This pattern is used frequently in builtin modules and types.

Here is an example:

from typing import overload

class bytes:
    ...
    @overload
    def __getitem__(self, i: int) -> int: ...
    @overload
    def __getitem__(self, s: slice) -> bytes: ...

So we basically have a definition of the same function __getitem__ with different arguments.

And another example:

from typing import Callable, Iterable, Iterator, Tuple, TypeVar, overload

T1 = TypeVar('T1')
T2 = TypeVar('T2')
S = TypeVar('S')

@overload
def map(func: Callable[[T1], S], iter1: Iterable[T1]) -> Iterator[S]: ...
@overload
def map(func: Callable[[T1, T2], S],
        iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterator[S]: ...
# ... and we could add more items to support more than two iterables

Here we have a definition of the same function map with a different number of arguments.

Answer 3

A single function is not allowed to have only leading optional parameters:

8.6. Function definitions

[...] If a parameter has a default value, all following parameters up until the “*” must also have a default value — this is a syntactic restriction that is not expressed by the grammar.

Note this excludes keyword-only parameters, which never receive arguments by position.

---

If desired, one can emulate such behaviour by manually implementing the argument to parameter matching. For example, one can dispatch based on arity, or explicitly match variadic arguments.

def leading_default(*args):
    # match arguments to "parameters"
    *_, low, high, size = 0, *args
    print(low, high, size)

leading_default(1, 2)     # 0, 1, 2
leading_default(1, 2, 3)  # 1, 2, 3

A simple form of dispatch achieves function overloading by iterating signatures and calling the first matching one.

import inspect


class MatchOverload:
    """Overload a function via explicitly matching arguments to parameters on call"""
    def __init__(self, base_case=None):
        self.cases = [base_case] if base_case is not None else []

    def overload(self, call):
        self.cases.append(call)
        return self

    def __call__(self, *args, **kwargs):
        failures = []
        for call in self.cases:
            try:
                inspect.signature(call).bind(*args, **kwargs)
            except TypeError as err:
                failures.append(str(err))
            else:
                return call(*args, **kwargs)
        raise TypeError(', '.join(failures))


@MatchOverload
def func(high, size):
    print('two', 0, high, size)


@func.overload
def func(low, high, size):
    print('three', low, high, size)


func(1, 2, size=3)    # three 1 2 3
func(1, 2)            # two 0 1 2
func(1, 2, 3, low=4)  # TypeError: too many positional arguments, multiple values for argument 'low'