12

The Python library pure_protobuf forces its users to use dataclasses, and decorate them with another decorator:

# to be clear: these two decorators are library code (external)
@message
@dataclass
class SearchRequest:
    query: str = field(1, default='')
    page_number: int32 = field(2, default=int32(0))
    result_per_page: int32 = field(3, default=int32(0))

This @message decorator assigns the SearchRequest instance a method called dumps:

SearchRequest(
    query='hello',
    page_number=int32(1),
    result_per_page=int32(10),
).dumps() == b'\x0A\x05hello\x10\x01\x18\x0A'

In my application code, I have a specific use-case where I need to pass an object that has the dumps() method. It can be a pure_protobuf Message instance like above, or it can be any other type, so long as it implements dumps().

It's working fine for classes that I've defined myself and implement the dumps() "interface", but for pure_protobuf data-classes, it keeps complaining that they have no attribute dumps().

What is making this more challenging is I'm not defining these pure_protobuf data-classes myself, these will be defined by clients of my library, so I can't simply do something (silly) like:

@message
@dataclass
class SearchRequest:
    query: str = field(1, default='')
    page_number: int32 = field(2, default=int32(0))
    result_per_page: int32 = field(3, default=int32(0))
    
    def dumps(self):
       self.dumps() # that is Message.dumps from the decorator

Am I out of options?

zerohedge
  • 3,185
  • 4
  • 28
  • 63

1 Answers1

8

Unfortunately, you're really out of solutions here, because you need (no matter that it's external, it's not really important) message decorator to return Intersection (or Meet in terms of types theory) of input class and protocol with 4 methods (dump, dumps, load, loads). It is not in python type system yet and is not implemented as type checker extension. See discussion regarding Intersection in mypy issue.

The most interesting thing is that you could use pytype instead and leave message unannotated, according to this tutorial. If using another type checker is an option for you, you could declare your own message version:

from typing import IO, Protocol, TYPE_CHECKING
from pure_protobuf.dataclasses_ import message as _message

class MessageMixin(Protocol):
    def dumps(self) -> bytes: ...
    def dump(self, io: IO) -> None: ...
    # Other definitions can go here

def message(cls):
    if TYPE_CHECKING:  # tweak
        return type(cls.__name__, (MessageMixin, cls), {})
    else:  # actually run on runtime
        return _message(cls)

Then users of your library can safely use this message implementation, because in fact it just wraps existing method for type checking purpose, not affecting runtime. So if they don't, they just have mypy errors (or not, if they do not use type checkers), but runtime is not affected.

However, once again, it does not work for mypy.

STerliakov
  • 4,983
  • 3
  • 15
  • 37
  • Nice solution, thanks. I’m upvoting because of its usefulness (+ learnt something new!), but I’m looking for mypy solutions (if those exist). – zerohedge Oct 12 '22 at 13:27
  • 2
    The full solution here, I imagine, is to write a mypy plugin. I know Django does this to support many of its complicated idioms that don't fit into mypy natively, but I imagine it's a huge undertaking in itself. – Silvio Mayolo Oct 12 '22 at 21:36
  • This is, unfortunately, often the cost of retrofitting static typing into Python. At least for now and the foreseeable future. – CrazyChucky Oct 12 '22 at 22:26