Polymorphism in Callablle under python type checking (Pylance)

Question

For my code I have an aggregate class that needs a validation method defined for each of the subclasses of base class BaseC, in this case InheritC inherits from BaseC.

The validation method is then passed into the aggregate class through a register method.

See the following simple example

from typing import Callable


class BaseC:
    def __init__(self) -> None:
        pass
    
class InheritC(BaseC):
    def __init__(self) -> None:
        super().__init__()

    @classmethod
    def validate(cls, c:'InheritC') ->bool:
        return False

class AggrC:
    def register_validate_fn(self, fn: Callable[[BaseC], bool])-> None:
        self.validate_fn = fn

ac = AggrC()
ic = InheritC()
ac.validate_fn(ic.fn)

I added type hints on the parameter for registering a function, which is a Callable object Callable[[BaseC], bool] since potentially there will be several other validation methods which is defined for each class inherited from BaseC.

However, pylance doesn't seem to recognize this polymorphism in a Callable type hint and throws a warning (I set up my VScode to type check it) that said

Argument of type "(c: InheritC) -> bool" cannot be assigned to parameter "fn" of type "(BaseC) -> bool" in function "register_fn"
  Type "(c: InheritC) -> bool" cannot be assigned to type "(BaseC) -> bool"
    Parameter 1: type "BaseC" cannot be assigned to type "InheritC"
      "BaseC" is incompatible with "InheritC" Pylance(reportGeneralTypeIssues)

I don't see where I made an mistake in design, and I don't want to simply ignore the warning.

Can any one explain why this is invaid? Or is it just simply a bug from pylance

I'm using python version 3.8.13 for development.

Answer 1

I'll be using the below sample code, where I've fixed a couple of bugs:

from typing import Callable


class BaseC:
    def __init__(self) -> None:
        pass

class InheritC(BaseC):
    def __init__(self) -> None:
        super().__init__()

    @classmethod
    def validate(cls, c:'InheritC') ->bool:
        return False

class AggrC:
    def register_validate_fn(self, fn: Callable[[BaseC], bool])-> None:
        self.validate_fn = fn

ac = AggrC()
ic = InheritC()
ac.register_validate_fn(ic.validate)

This Python runs without errors, but still produces the same error you're seeing when run through a type checker (in my case, MyPY):

$ mypy stackoverflow_test.py 
stackoverflow_test.py:21: error: Argument 1 to "register_validate_fn" of "AggrC" has incompatible type "Callable[[], bool]"; expected "Callable[[BaseC], bool]"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

This is a subtle issue, which is easy to overlook: it's an issue with contravariance of parameter types.

The reason this is easy to overlook is because most object-oriented tutelage focuses on classes and objects, and doesn't really discuss functions as being types with inheritance. Indeed, most languages with object-oriented language features don't support declaring a function as inheriting from another function!

Lets reduce this as much as possible:

from typing import Callable

class Parent:
    def foo(self):
        pass

class Child(Parent):
    def bar(self):
        pass
    
def takesParent(parameter: Parent):
    parameter.foo()
    
def takesChild(parameter: Child):
    parameter.bar()
    
def takesFunction(function: Callable):
    # What should the signature of `function` be to support both functions above?
    pass

How should you define function: Callable to make it compatible with both functions?

Lets take a look at what takesFunction could do, which would be valid for both functions:

def takesFunction(function: Callable):
    child = Child()
    function(child)

This function should work if you pass either function, because takesParent will call child.foo(), which is valid; and takesChild will call child.bar(), which is also valid.

OK, how about this function?

def takesFunction(function: Callable):
    parent = Parent()
    function(parent)

In this case, function(parent) can only work with takesParent, because if you pass takesChild, takesChild will call parent.bar() - which doesn't exist!

So, the signature that supports passing both functions looks like this:

def takesFunction(function: Callable[[Child], None]):

Which is counter-intuitive to many people.

The parameter function must be type-hinted as taking the most specific parameter type. Passing a function with a less specific parameter - a superclass - is compatible, but passing one with a more specific parameter isn't.

This can be a difficult topic to understand, so I'm sorry if I didn't make it very clear, but I hope this answer helped.