3

Let's say I have the following (common, there are similar SO questions, e.g. about filtering set[re.Match | None]):

from collections.abc import Sequence

def foo(items: Sequence[str | None]) -> list[str]:
    if all(items):
        return [item + '!' for item in items]  # E: Unsupported left operand type for + ("None")  [operator]
    raise ValueError

This fails, claiming that None + str is an invalid operation.

This is obvious, given that in builtins.py of typeshed we currently have a simple definition:

def all(__iterable: Iterable[object]) -> bool: ...

However, let's pretend that we are typeshed authors (or are using custom typeshed with mypy) and tweak the definition:

from collections.abc import Iterable
from typing import TypeGuard, TypeVar

_T = TypeVar('_T')
def all(__iterable: Iterable[_T | None]) -> TypeGuard[Iterable[_T]]: ...

Now the code above typechecks properly! Here's the playground link with this solution.

Usually we (moreover, I did it myself) answer that it's too much of magic for type checker to support such kind of inference - but that seems to be wrong? It's just about definition of builtin function.

What are the drawbacks of such definition? I understand that it is probably "more than at is" (I mean that all is like a one-size-fits-all solution, nobody thinks about it as about such type guard, and None is only one example of filtered out values) - but hey, it (I guess) doesn't harm, and simplifies some code pieces greatly.

Moreover, it allows for simple testing of several variables as once:

def foo(a: str | None, b: str | None) -> str:
    together = [a, b]
    if all(together):
        a, b = together
        return a + b
    raise ValueError

And yes, the above typechecks too with my all definition.

To link, I opened an issue for this in typeshed repo.

STerliakov
  • 4,983
  • 3
  • 15
  • 37
  • I looked at the PEP tracker for TypeGuard (https://github.com/python/typeshed/issues/5406) and it seems like you're actually OK to start implementing TypeGuard in typeshed. I guess the answer is: nobody's suggested it, so make a PR and see what happens; see if any of `pytype`, `mypy`, `pyright`, and `pyre` have catastrophic failures :) – dROOOze Feb 18 '23 at 12:23
  • Hm, but typeshed [already uses TypeGuard in 13 definitions](https://github.com/python/typeshed/search?q=TypeGuard). – STerliakov Feb 18 '23 at 12:27
  • 2
    On second thoughts, since higher kinded types aren’t supported yet, it probably isn’t that useful to turn e.g. a `dict[str | None, object]` into an `Iterable[str]`. Ideally you’d pass in a type bounded to `Iterable[T]`, which would make the type guard more useful. – dROOOze Feb 18 '23 at 12:45
  • 1
    Yeah, I finally discovered that too, seems that upcasting with TypeGuard is a popular problem. If you post it as answer, I'll happily accept it (I already commented it on github issue). – STerliakov Feb 18 '23 at 13:07

0 Answers0