Kwargs in a Protocol implementer: what is a valid signature?

Question

My question is simple. I have this protocol:

from typing import Protocol

class LauncherTemplateConfig(Protocol):
    def launch_program_cmd(self, **kwargs) -> list[str]:
        pass

And this implementation of the protocol, which I would expect mypy passes, but it does not:

from typing import Optional
from pathlib import Path

class MyLauncherTemplateConfig:
    def launch_program_cmd(
        self, some_arg: Optional[Path] = None, another_arg=1
    ) -> list[str]:

I would expect the parameters in MyLauncherTemplateConfig.launch_program_cmd to be compatible with **kwargsin the Protocol class.

Not sure if I am doing something wrong...

Alex Waygood · Accepted Answer · 2021-08-14T15:12:42.897

The general principle

If you want MyPy to accept that a certain class implements the interface defined in a Protocol, a relevant method in the concrete implementation must be no less permissive in the arguments it will accept than the abstract version of that method as defined in the Protocol. This is consistent with other principles of object-oriented programming such as the Liskov Substitution Principle.

The specific issue here

Your Protocol defines an interface in which the launch_program_cmd method can be called with any keyword-arguments, and not fail at runtime. Your concrete implementation does not satisfy this interface, as any keyword arguments other than some_arg or another_arg will cause the method to raise an error.

Possible solution

If you want MyPy to declare your class as a safe implementation of your Protocol, you have two options. You can either adjust the signature of the method in the Protocol to be more specific, or adjust the signature of the method in the concrete implementation to be more generic. In the case of the latter, you might do it like this:

from typing import Any, Protocol, Optional
from pathlib import Path

class LauncherTemplateConfig(Protocol):
    def launch_program_cmd(self, **kwargs: Any) -> list[str]: ...


class MyLauncherTemplateConfig:
    def launch_program_cmd(self, **kwargs: Any) -> list[str]:
        some_arg: Optional[Path] = kwargs.get('some_arg')
        another_arg: int = kwargs.get('another_arg', 1)
        # and then the rest of your method

By using the dict.get method, we can retain the default values in your implementation, but do it in a way that sticks to the generic signature of the method that was declared in the Protocol

Thanks. This is the path I considered taking. I will accept since there is no other way since signatures are somewhat "static", actually. Otherwise it would violate that principle, yes. — Germán Diago, Aug 13 '21 at 23:17

score 0 · Answer 2 · answered Aug 13 '21 at 20:26

The key issue is that your permissiveness is upside down. Formally, it's because functions are contravariant in their inputs.

What you're promising with def launch_program_cmd(self, **kwargs) -> list[str]: is "This method will be able to take any set of keyword arguments."

By way of example, if someone writes

def launch_lunch(launcher: LauncherTemplateConfig):
    launcher.launch_program_cmd(food=["eggs", "spam", "spam"])

then according to the definition of LauncherTemplateConfig that should be allowed.

But if you try to call that method with an instance of MyLauncherTemplateConfig then it will crash because it doesn't know what to do with the food parameter. So MyLauncherTemplateConfig is not a valid subtype of LauncherTemplateConfig

I suspect that what you're intending to convey is more like "This method will exist, but I don't know what arguments it will take." However, that's not really something that MyPy is set up to express. The basic reason is that it's not very useful: there's not a lot you can do with a promise that a method will exist but you don't know how to call it!

(Note: the opposite direction is allowed. If your Protocol specified that you must be able to take some_arg and another_arg and your implementation was able to handle anything at all, that would be allowed. But generally, you'd want your protocol to guide you in what you'd actually want to take.)

In fact I do not know the arguments, that is why I use **kwargs in the base class, meaning: "my protocol-implementing class has a signature with any number of keywor arguments". These are checked at run-time later down the road, since I consume those arguments dynamically. — Germán Diago, Aug 13 '21 at 21:43
That's your issue then. If you were saying that the implementing class could handle anything, you're be OK. Something like `def launch_program_cmd(self, *args: Any, **kwargs: Any):` in the Protocol and `def launch_program_cmd(self, some_arg: Optional[Path] = None, another_arg=1, *args: Any, some_kwarg: float=1.2, **kwargs: Any):` in the implementation. — Josiah, Aug 14 '21 at 13:38
Note that if you don't have *args in the protocol you could only add keyword arguments, and if you don't have **kwargs in the protocol you could only add positional ones. — Josiah, Aug 14 '21 at 13:39

Kwargs in a Protocol implementer: what is a valid signature?

2 Answers2