How to make python typing recognize subclasses as valid types when it expects their parent class?

Question

Here is a minimal example of what I need to do:

from typing import Callable, Any


class Data:
    pass


class SpecificData(Data):
    pass


class Event:
    pass


class SpecificEvent(Event):
    pass


def detect_specific_event(data: SpecificData, other_info: str) -> SpecificEvent:
    return SpecificEvent()


def run_detection(callback: Callable[[Data, Any], Event]) -> None:
    return


run_detection(detect_specific_event)

Now I get a warning:

Expected type '(Data, Any) -> Event', got '(data: SpecificData, other_info: str) -> SpecificEvent' instead

To me it seems like this warning doesn't make sense, as SpecificData and SpecificEvent are subtypes of Data and Event respectively, so everything should be fine. Is there a way to make this work as I expect? My idea is to be able to then have something like:

class OtherSpecificData(Data):
    pass


class OtherSpecificEvent(Event):
    pass


def detect_other_event(data: OtherSpecificData, other_info: str) -> OtherSpecificEvent:
    return OtherSpecificEvent()

run_detection(detect_other_event)

so the run_detection function is as general as possible. Right now this gives the same warning as above.

The problem here is your argument type in `run_detection` implies that the callable passed should be able to work on `Data` and all its subclasses, but then you pass it a callable that says it cannot work on `Data`, it can only work on `SpecificData`. For example, suppose `detect_specific_event` used an attribute in `SpecificData` that wasn't available in the parent class. But `run_detection` is told that the callback it should expect won't do that; it is told it will work on `Data` and all its subclasses. So then why does the function it is passed require `SpecificData`? — alkasm, Mar 01 '20 at 07:42
It seems like you are expecting the parent class to act as a union of its subclasses, but it cannot, because the subclasses may have functionality which the parent does not. If `run_detection` can call the callback regardless of the datatype, then you are asking the function to require a callable to be more specific (only operate on the base class) than you want (operate on any subclass). As far as `run_detection` is concerned, the relation between the classes are irrelevant. An easy fix is to just use a `Union` of your subtypes. Otherwise, look up "composition over inheritance". — alkasm, Mar 01 '20 at 07:54
@Alkasm I mostly agree, but... the 2 functions here could work, with LSP in mind, if further data,**which are not appearing here** were all SpecificData. But the OP’s code only shows wiring up `run_detection` which is what they asked about here, not how it gets Datas. If next a `SpecificData` only generator is spun up then it would work. But not if it emits `Data` events. — JL Peyret, Mar 01 '20 at 18:09
@JLPeyret indeed my comments make an assumption about what the OP intends to do, which may or may not be true. But since OP states at the end "so `run_detection` is as general as possible", the main thing to hammer home here is that `run_detection` expecting a callable on the base class is *more* restrictive, not more general. — alkasm, Mar 02 '20 at 01:30

score 2 · Accepted Answer · answered Feb 28 '20 at 08:49

2

Parameter sub-typing is opposite direction with return sub-typing.

Return value is assigned from callee to caller.
Parameter value is assigned from caller to callee.

And assign value should be more specific than variable's expected type. For example:

data: Data = SpecificData()  # okay
data: SpecificData = Data()  # not okay

So you should do:

from typing import Callable, Any


class Data:
    pass


class SpecificData(Data):
    pass


class Event:
    pass


class SpecificEvent(Event):
    pass


def detect_specific_event(data: Data, other_info: str) -> SpecificEvent:
    return SpecificEvent()


def run_detection(callback: Callable[[SpecificData, Any], Event]) -> None:
    return


run_detection(detect_specific_event)

answered Feb 28 '20 at 08:49

Boseong Choi

2,566
9
22

Thanks, that clears up why I'm getting the warning. Does that mean there is no way to define `run_detection` so that it can accept both `detect_specific_event` and `detect_other_event` from my example? – Petar Chernev Feb 28 '20 at 09:01
I think so. Both functions need different specific types so you cannot generalize them. I suggest to change `detect_specific_event`'s parameter type to `Data, Any` if you need generalization. – Boseong Choi Feb 28 '20 at 09:10
hum, sorry, without knowing that much about typing, that makes little sense conceptually. from its name, I would expect that `detect_specific_event` wants a `SpecificData` and might have problems with more generic versions. yet, to make typing happy your function signatures convey exactly the opposite meaning to a human reader. – JL Peyret Feb 28 '20 at 20:43
1

@PetarChernev Create `T = TypeVar("T", bound=Data)`, then you can type `run_detection` as `(callback: Callable[[T, Any], Event]) -> None`. This accepts both `detect_specific_event` and `detect_other_event`. – ringo Jan 11 '23 at 08:16

JL Peyret · Answer 2 · 2020-03-01T18:20:23.260

Took me a while to remember which piece of typing to use, but IMHO you want to use cast

Unlike its use in other languages, cast(x,y) doesn't do anything, but it does tell typing consider y as a type x. runtime, it's a no-op, just returns y.

Like compiled languages, if I read it, I would pay special attention to the code: is that really going to work at runtime? are the data types actually going to be correct?:

The duplicate closure with the LSP remark is appropriate if you can’t guarantee that whatever is generating data later is only going to hand out SpecificDatas. If you can, then casting would be OK. Your minimal example is missing that bit, but if you had shown what actual data was passing through print(data) then we’d known if LSP applied.

from typing import Callable, Any, cast


class Data:
    pass


class SpecificData(Data):
    pass


class Event:
    pass


class SpecificEvent(Event):
    pass


def detect_specific_event(data: SpecificData, other_info: str) -> SpecificEvent:
    return SpecificEvent()


def run_detection(callback: Callable[[Data, Any], Event]) -> None:
    return


run_detection(cast((Callable[[Data, Any], Event]),detect_specific_event))

Here, you've basically told typing, "accept my word for it" that detect_specific_event is a Callable[[Data, Any], Event]).

outputs of runs and type checks:

$ mypy test2.py
Success: no issues found in 1 source file
$ python test2.py
(venv)$   well your code says nothing.

change cast to the actual sig:

run_detection(cast((Callable[[SpecificData, Any], SpecificEvent]),detect_specific_event))

(venv) $@so.mypy$ mypy test2.py
Argument 1 to "run_detection" has incompatible type "Callable[[SpecificData, Any], SpecificEvent]"; expected "Callable[[Data, Any], Event]"
Found 1 error in 1 file (checked 1 source file)
$ python test2.py 
$ well your code says nothing.

How to make python typing recognize subclasses as valid types when it expects their parent class?

2 Answers2