How to use static type checking using Dict with different value types in Python 3.6?

Question

Trying to use static types in Python code, so mypy can help me with some hidden errors. It's quite simple to use with single variables

real_hour: int = lower_hour + hour_iterator

Harder to use it with lists and dictionaries, need to import additional typing library:

from typing import Dict, List
hour_dict: Dict[str, str] = {"test_key": "test_value"}

But the main problem - how to use it with Dicts with different value types, like:

hour_dict = {"test_key": "test_value", "test_keywords": ["test_1","test_2"]}

If I don't use static typing for such dictionaries - mypy shows me errors, like:

len(hour_dict['test_keywords'])
- Argument 1 to "len" has incompatible type

So, my question: how to add static types to such dictionaries? :)

[They're still working on that.](https://github.com/python/typing/issues/28) — user2357112, Dec 28 '17 at 19:59
If all of the types that the dict could contain are known, you could use `Union`. e.g. `Dict[str, Union[list, str]]`. However, this doesn't ensure that particular keys are always of a specified type, it just allows values to be either (e.g.) strings or lists. — kindall, Dec 28 '17 at 20:01
Related: [Python 3 dictionary with known keys typing](https://stackoverflow.com/q/44225788/7851470) — Georgy, Oct 10 '19 at 15:19

Jonathan Eunice · Accepted Answer · 2017-12-28T20:20:43.233

23

You need a Union type, of some sort.

from typing import Dict, List, Union

# simple str values
hour_dict: Dict[str, str] = {"test_key": "test_value"}

# more complex values
hour_dict1: Dict[str, Union[str, List[str]]] = {
    "test_key": "test_value", 
    "test_keywords": ["test_1","test_2"]
}

In general, when you need an "either this or that," you need a Union. In this case, your options are str and List[str].

There are several ways to play this out. You might, for example, want to define type names to simplify inline types.

OneOrManyStrings = Union[str, List[str]]

hour_dict2: Dict[str, OneOrManyStrings] = {
    "test_key": "test_value", 
    "test_keywords": ["test_1","test_2"]
}

I might also advise for simplicity, parallelism, and regularity to make all your dict values pure List[str] even if there's only one item. This would allow you to always take the len() of a value, without prior type checking or guard conditions. But those points are nits and tweaks.

edited Dec 28 '17 at 20:20

answered Dec 28 '17 at 20:06

Jonathan Eunice

21,653
6
75
77

Additional question: how to use static typing with datetime objects? – sortas Dec 28 '17 at 20:35
1

@sortas Once a type is defined or imported, it can be used in typed Python just like a builtin such as `int` or `str`. So after `from datetime import datetime` you can just say `right_now: datetime = datetime.utcnow()`. Or you can use `datetime` in `Union`s or as the subtype of `List` and `Dict`, or otherwise used just as primitive types can be used. – Jonathan Eunice Dec 28 '17 at 20:56
Enuice Ok, understand, thanks. Still, mypy doesn't like Union: I define dict as `order_data: Dict[str, Union[str, int, List[str]]]`, but when I try to take data from it (`keywords_to_load: List[str] = order_data['keywords']`) mypy says "Incompatible types, expression has type `Union[str, int, List[str]]`, variable has type `List[str]`. Am I doing something wrong? I mean, it's not really a problem, still want to fix as may errors as possible :) – sortas Dec 28 '17 at 21:03
1

@sortas The problem is that mypy cannot *guarantee* the safety of that operation. Yes, it might be a `List[str]` value retrieved from `order_data['keywords']`. But it might equally be a `str` or an `int`. Mypy can't know, therefore it's not passing the checks. Try [something like this](https://gist.github.com/jonathaneunice/8a1bf79bd34a72da0f3c33504b481174) instead. That is a type equation mypy can guarantee. – Jonathan Eunice Dec 28 '17 at 21:19
when you use python 3.11, you can create a union easily by doing e. g. [str|int], so with the | character. – Jasper Bart Aug 22 '23 at 11:33

score 5 · Answer 2 · answered Jan 01 '18 at 08:04

While using Union is indeed one way of doing it, a more precise solution would be to use the (currently experimental) TypedDict type, which lets you assign specific types per each string key.

In order to use this type, you must first install the mypy_extensions 3rd party library using pip. You can then do the following:

from typing import List
from mypy_extensions import TypedDict

MyDictType = TypedDict('MyDictType', {
        'test_key': str, 
        'test_keywords': List[str],
})

hour_dict: MyDictType = {
    "test_key": "test_value", 
    "test_keywords": ["test_1","test_2"]
}

Note that we need to explicitly denote hour_dict of being of type MyDictType. A slightly cleaner way of doing this is to use MyDictType as a constructor -- at runtime, MyDictType(...) is exactly equivalent to doing dict(...), which means the below code behaves exactly identically to the above:

hour_dict = MyDictType(
    test_key="test_value", 
    test_keywords=["test_1","test_2"]
)

Finally, note that there are a few limitations to using TypedDict:

It's useful only when the dict will contain specific keys with types that are all known at compile time -- you should use regular Dict[...] if you expect a truly dynamic dict.
The keys must all be strings.
At time being, this type is understood only by mypy (though I understand there are plans to eventually add TypedDict to PEP 484 once it's a little more battle-tested, which would mean any PEP 484 compliant typechecker would be required to support it).

(TypedDict was designed to make it easier to work with JSON blobs/dicts when writing serialization/deserialization logic, which is the reason for these constraints.)

How to use static type checking using Dict with different value types in Python 3.6?

2 Answers2

Linked