Recursive type annotations

Question

I'm trying to introduce static type annotations to my codebase where applicable. One case is when reading a JSON, the resulting object will be a dictionary keyed by strings, with values of one of the following types:

bool
str
float
int
list
dict

However the list and dict above can contain that same sort of dictionary, leading to a recursive definition. Is this representable in Python3's type structure?

Does `JSONVal = Union[blah, blah, blah, List['JSONVal'], Dict[str, 'JSONVal']]` and annotating JSON values as `JSONVal` work? — user2357112, Dec 05 '18 at 19:01
I see a [mypy issue](https://github.com/python/mypy/issues/731) about recursive types that seems to suggest it's not possible, although I haven't read the whole thing yet. — user2357112, Dec 05 '18 at 19:03
@user2357112 that issue link is very helpful. One comment near the bottom seems to get to the end of the problem: after several workarounds for JSON specifically, they note that "`Dict[str, Any]` is sadly the name of the game most of the time." — Adam Smith, Dec 05 '18 at 19:07
One alternative to explore is [TypedDict](https://mypy.readthedocs.io/en/latest/more_types.html#typeddict). They're useful if you know exactly what the JSON you're receiving looks like: for example, when calling some API that always returns JSON structured in a set way. Also, if you want to validate your JSON at runtime, I recommend using `Dict[str, object]` over `Dict[str, Any]`: it'll help you confirm you're adding isinstance checks and stuff in the right places. If you don't care about runtime validation, either casting to a TypedDict or using `Dict[str, Any]` is probably the right choice. — Michael0x2a, Dec 06 '18 at 00:52
@Michael0x2a that's a good reference, but no -- I don't know anything about the JSON I'm receiving during runtime, so I can't validate it in any meaningful way. — Adam Smith, Dec 06 '18 at 01:02

user2357112 · Accepted Answer · 2022-11-28T05:43:26.007

As of mypy 0.990, mypy finally supports recursive type annotations, using the natural syntax:

from typing import Union, Dict, List

JSONVal = Union[None, bool, str, float, int, List['JSONVal'], Dict[str, 'JSONVal']]

d: JSONVal = {'a': ['b']}

mypy output:

Success: no issues found in 1 source file

Before 0.990, this would produce an error reporting a lack of recursive type support:

$ mypy asdf.py
asdf.py:3: error: Recursive types not fully supported yet, nested types replaced with "Any"

On such versions, Dict[str, Any] would be the way to go.

You can also use mutually recursive type aliases now, so you can do things like

from typing import Union, Dict, List

JSONVal = Union[None, bool, str, float, int, 'JSONArray', 'JSONObject']
JSONArray = List[JSONVal]
JSONObject = Dict[str, JSONVal]

d: JSONObject = {'a': ['b']}

score 5 · Answer 2 · answered Oct 26 '22 at 11:55

5

Support for recursive types is now in Mypy.

As of October 2022 the implementation is provisional. You can enable it by adding the enable_recursive_aliases = true flag to pyproject.toml.

Starting from version 0.990 this will be enabled by default. Source.

answered Oct 26 '22 at 11:55

theberzi

2,142
3
20
34

0.990 is now out and recursive aliases are enabled by default. If you set the above flag you will get a deprecation warning. – theberzi Nov 09 '22 at 12:37

score -2 · Answer 3 · answered Mar 29 '22 at 15:33

For more recent versions of mypy, this comment to the mentioned MyPy issue tracker suggests a partially working (but a bit convoluted) way of doing this using protocols, as long a using TypeVar is not required:

from __future__ import annotations

from collections.abc import Iterator
from typing import TypeVar, Protocol, overload, Any, TYPE_CHECKING

_T_co = TypeVar("_T_co")

class _RecursiveSequence(Protocol[_T_co]):
    def __len__(self) -> int: ...
    @overload
    def __getitem__(self, __index: int) -> _T_co | _RecursiveSequence[_T_co]: ...
    @overload
    def __getitem__(self, __index: slice) -> _RecursiveSequence[_T_co]: ...
    def __contains__(self, __x: object) -> bool: ...
    def __iter__(self) -> Iterator[_T_co | _RecursiveSequence[_T_co]]: ...
    def __reversed__(self) -> Iterator[_T_co | _RecursiveSequence[_T_co]]: ...
    def count(self, __value: Any) -> int: ...
    def index(self, __value: Any, __start: int = ..., __stop: int = ...) -> int: ...


def func1(a: _RecursiveSequence[int]) -> int: ...

if TYPE_CHECKING:
    reveal_type(func1([1]))         # Revealed type is "builtins.int"
    reveal_type(func1([[1]]))       # Revealed type is "builtins.int"
    reveal_type(func1([[[1]]]))     # Revealed type is "builtins.int"
    reveal_type(func1((1, 2, 3)))   # Revealed type is "builtins.int"
    reveal_type(func1([(1, 2, 3)])) # Revealed type is "builtins.int"
    reveal_type(func1([True]))      # Revealed type is "builtins.int"

How do you define a JSON typehint in python with the above ? Your answer is a bit raw. Can you elaborate ? — vianmixt, Oct 02 '22 at 13:40

Recursive type annotations

3 Answers3

Linked

Related