Can Python type hints handle a generic class changing its own type in a method?

Question

I have an existing class that changes an important type upon a certain method call. It looks something like this:

class RememberLast:
    def __init__(self, last):
        self._last = last

    def set(self, new_last):
        self._last = new_last

    def get(self):
        return self._last

remember = RememberLast(5)
type(remember.get())  # int
remember.set('wow')
type(remember.get())  # str
remember.set(4.5)
type(remember.get())  # float

Ideally the type of remember would change from RememberLast[int] to RememberLast[str] and then to RememberLast[float]. Is there a way to represent this situation with type hints?

Returning self with a different type hint in set() isn't ideal because there are existing callers. For these existing callers that don't use the return value, the type would stay as RememberLast[int] even though the type was "destroyed" and isn't correct anymore.

The existing class I'm referring to is twisted.internet.defer.Deferred, which allows chaining callbacks. The type of the last return value becomes the parameter for the next callback. So the type of a Deferred can be thought of as the type of the last callback added to it.

if you know the all possible types then put them in union. like: def get(self) -> Union[int, str, float]: — S4eed3sm, Jan 02 '22 at 04:41
Type-hints are for *static* typing. If you change the type at runtime, that's by definition *not* static typing. — chepner, Jan 03 '22 at 01:13

Soto · Answer 1 · 2022-01-03T00:26:54.390

You can use a generic class to track the changing type.

Define two generics.

T = The current type
R = The type after we call set

from typing import TypeVar, Generic, Type

T = TypeVar("T")
R = TypeVar("R")


class RememberLast(Generic[T]):
    def __init__(self, last: T):
        self._last = last

    def set(self, new_last: R) -> "RememberLast[R]":
        return RememberLast(new_last)

    def get(self) -> T:
        return self._last

Note in order for this to work you will need to reassign the results of set back to remember. We need to do this because the results of set hold the new type hinting information.

# int
remember = RememberLast(5)  # RememberLast[int]
print(type(remember.get()))  # <class 'int'>

# str
remember = remember.set("abc")  # RememberLast[str]
print(type(remember.get()))  # <class 'str'>

# float
remember = remember.set(4.5)  # RememberLast[float]
print(type(remember.get()))  # <class 'float'>

but what if RememberLast is subclassed? and set is rewritten as `type(self)(new_last)` -- you'd like to go from SubClass[T] to SubClass[R] in the typing, not SubClass[T] to RememberLast[R] -- is this possible? — Michael Scott Asato Cuthbert, Dec 08 '22 at 01:42

BrokenBenchmark · Answer 2 · 2022-01-02T04:57:45.303

I'm going to assume that RememberLast must be able to handle things besides ints, strings, and floats (or more than a finite union of types), because otherwise you could just use Union[int, str, float].

I'm unfortunately pessimistic that you can use an annotation more specific than Any for two reasons:

mypy's own's documentation suggests using Any for dynamically typed code. If it was possible to use something more specific, they would have said so.
This StackOverflow question asks something similar, and the accepted answer is basically "refactor so you can explicitly define the expected types, or use Any".

score -3 · Answer 3 · answered Jan 02 '22 at 04:42

Unfortunately, type hints in python doesn't prevent the type from changing for example in your snippet if you used the type hints as bellow it will give the same output:

class RememberLast:
    def __init__(self, last: int):
        self._last = last

    def set(self, new_last: int):
        self._last = new_last

    def get(self):
        return self._last

remember = RememberLast(5)
type(remember.get())  # int
remember.set('wow')
type(remember.get())  # str
remember.set(4.5)
type(remember.get())  # float

There are two ways to force the methods to take only specific type:

Assert with isinstance:

class RememberLast:
    def __init__(self, last: int):
        assert isinstance(last, int)
        self._last = last

    def set(self, new_last: int):
        assert isinstance(last, int)
        self._last = new_last

    def get(self):
        assert isinstance(last, int)
        return self._last

remember = RememberLast(5)
type(remember.get())  # int
remember.set('wow')
type(remember.get())  # raises AssertionError
remember.set(4.5)
type(remember.get())  # raises AssertionError

use mypy with type hinting before you deploy your code to make sure that types are matching correctly

mypy my_python_script.py

this should be the output

my_python_script.py:26: error: Argument 1 to "set" of "RememberLast" has incompatible type "str"; expected "int"
my_python_script.py:28: error: Argument 1 to "set" of "RememberLast" has incompatible type "float"; expected "int"
Found 2 errors in 1 file (checked 1 source file)

This doesn't answer OP's question. The code is supposed to take advantage of dynamic typing, and your solution introduces changes which prevent the code from running. — BrokenBenchmark, Jan 02 '22 at 04:43

Can Python type hints handle a generic class changing its own type in a method?

3 Answers3