Python mypy static typing: Accept only arguments of Dict (with key str), Lists (but not of custom classes), and Nested Dicts and Lists

Question

I want a function to accept a YAML style argument, that can be either a list or a dict. e.g

foo({'a' : 1}), or foo([{'a' : 1}, {'b' : 2, 'c' : [1,2]}]), or foo({'a' : {'b' : 1}})

If it is a dict, the key has to be a string, and the value has to be a Union[int, float, str], a list of these, another dict with the same typing rules as itself, or a list of dicts with the same rules. If it is a list it can be a list of Union[int, float, str], a list of dicts, or a list of lists with the same rules as itself.

No custom classes should be allowed, and the key for any dict must be a string.

I could simply do:

def foo(yaml_args : Union[List[Any], Dict[str, Any]):
    pass

But I want to avoid using Any.

My best attempt so far is:

from typing import Dict, List, Union

ScalarPod = Union[int, float, str]
Pod = Union[ScalarPod, List[ScalarPod]]

NestedDict = Dict[str, Union[
    Pod,
    Dict[str, Union[
        Pod,
        Dict[str, Union[
            Pod,
            Dict[str, Pod]
        ]]
    ]]
]]

def foo(yaml_args : NestedDict):
    print(str(yaml_args))
    
foo({
        'number' : 1,
        'number_list' : [1, 2, 3],
        'dict_a' : 
        {
            'number_list_b' : [2,3,4],
            'number_b' : 3 
        },
        'dict_b' :
        {
            'nested_dict' :
            {
                'number' : 1,
                'number_b' : 2,
            }
        },
        'dict_c' :
        {
            'nested_dict' :
            {
                'nested_2_dict' : 
                {
                    'number' : 1
                }
            }
        },
        'dict_list' : [
            {
                'a' : 1
            },
            {
                'b' : 2
            }
        ]
    }
)

This works apart from the list of dicts that I cant seem to fix.

It also has the disadvantage that it is limited to a max depth of 3 nested dicts, but this isn't a game killer. I would prefer if it had unlimtied depth, but it is not a requirement.

How could I achieve this? Thanks in advance!

The inability to handle recursive types is a know limitation of `mypy`, IIRC. — chepner, Jul 14 '20 at 15:53
Ideally, you would want something like `DataDict = Dict[str,Union[Data,'DataDict']]`. — chepner, Jul 14 '20 at 15:55
As for recursive types, see [How can I make a recursive Python type defined over several aliases?](https://stackoverflow.com/questions/58377361/how-can-i-make-a-recursive-python-type-defined-over-several-aliases). Can't say yet why the (non-recursive) manual nesting is rejected. — MisterMiyagi, Jul 14 '20 at 15:55
@chepner You're spot on. Recursive types is exactly what I want. It sounds like it is in the road map though so I guess I just have something to look forward to. — Blue7, Jul 14 '20 at 15:57

score 0 · Answer 1 · answered Jul 14 '20 at 15:50

0

Mypy is a static type checker. Your input variable can be 2 types as you have just indicated in your first sentence. Therefore it does not have a static (non changing) type.

Potential Solution: separate the Input variable into two variable with different types and add a default value of None. Then in your function check which variable is not none and then you can use that one.

answered Jul 14 '20 at 15:50

Bhargav Tarpara

27
5

The ``Union`` type exists to allow multiple (exclusive) types for one variable. This is exactly the "either a list or a dict" relation in the first sentence. – MisterMiyagi Jul 14 '20 at 15:52
Static typing does not preclude recursive data types. – chepner Jul 14 '20 at 15:58

score 0 · Answer 2 · answered Jul 14 '20 at 17:36

I have found a way to make it easier to support deeper nested dicts, but still not lists of dicts:

ScalarPod = Union[int, float, str]
Pod = Union[ScalarPod, List[ScalarPod]]
PodDict = Dict[str, Pod]

T = TypeVar("T")

NestedDict = Dict[str, Union[
    Pod,
    T
]]

NestedDict = NestedDict[NestedDict[NestedDict[NestedDict[NestedDict[NestedDict[PodDict]]]]]]

a : NestedDict = {
        'number' : 1,
        'number_list' : [1.1, 2, 3],
        'dict_a' : 
        {
            'number_list_b' : [2,3,4],
            'number_b' : 3 
        },
        'dict_b' :
        {
            'nested_dict' :
            {
                'number' : 1,
                'number_b' : 2,
            }
        },
        'dict_c' :
        {
            'nested_dict' :
            {
                'nested_2_dict' : 
                {
                    'number' : 1
                }
            }
        }
    }

Python mypy static typing: Accept only arguments of Dict (with key str), Lists (but not of custom classes), and Nested Dicts and Lists

2 Answers2