Handling loads of different message-types at runtime in an elegant way

Question

In order to be able to handle large amounts of different request types I created a .proto file like this:

message Message
{
   string typeId = 1;
   bytes message = 2;
}

I added the typeId so that one knows what actual protobuf bytes represents. (Self-describing)

Now my problem is handling that different "concrete types" in an elegant way. (Note: All works fine if I simple use a switch-case-like approach!)

I thought about a solution like this:

1) Have a trait the different handlers have to implement, e.g.:

trait Handler[T]
{
  def handle(req: T): Any
}

object TestHandler extends Handler[Test]
{
  override def handle(req: Test): String =
  {
    s"A success, $req has been handled by TestHandler
  }
}

object OtherHandler extends Handler[Other]
{
  override def handle(req: Other): String =
  {
    s"A success, $req has been handled by OtherHandler
  }
}

2) provide some kind of registry to query the right handler for a given message:

val handlers = Map(
    Test -> TestHandler,
    Other -> OtherHandler
  )

3) If a request comes in it identifies itself, so we need another Mapper:

val reqMapper = Map(
  "Test" -> Test
  "Other" -> Other
)

4) If a request comes in, handle it:

val request ...
// Determine the requestType
val requestType = reqMapper(request.type) 
// Find the correct handler for the requestType
val handler = handlers(requestType)
// Parse the actual request
val actualRequest = requestType.parse(...) // type of actualRequest can only be Test or Other in our little example

Now, until here everything looks fine and dandy, but then this line breaks my whole world:

handler.handle(actualRequest)

It leads to:

type mismatch; found : com.trueaccord.scalapb.GeneratedMessage with Product with com.trueaccord.scalapb.Message[_ >: tld.test.proto.Message.Test with tld.test.proto.Message.Other <: com.trueaccord.scalapb.GeneratedMessage with Product] with com.trueaccord.lenses.Updatable[_ >: tld.test.proto.Message.Other with tld.test.proto.Message.Test <: com.trueaccord.scalapb.GeneratedMessage with Product]{def companion: Serializable} required: _1

As far as I understand - PLEASE CORRECT ME HERE IF AM WRONG - the compiler cannot be sure here, that actualRequest is "handable" by a handler. That means it lacks the knowledge that the actualRequest is definitely somewhere in that mapper AND ALSO that there is a handler for it.

It's basically implicit knowledge a human would get, but the compiler cannot infer.

So, that being said, how can I overcome that situation elegantly?

rogue-one · Answer 1 · 2017-02-04T04:46:35.623

1

your types are lost when you use a normal Map. for eg

object Test{}
object Other{}
val reqMapper = Map("Test" -> Test,"Other" -> Other)
reqMapper("Test")
res0: Object = Test$@5bf0fe62 // the type is lost here and is set to java.lang.Object

the most idomatic way to approach this is to use pattern matching

request match {
  case x: Test => TestHandler(x)
  case x: Other => OtherHandler(x)
  case _ => throw new IllegalArgumentException("not supported")
}

if you still want to use Maps to store your type to handler relation consider HMap provided by Shapeless here

Heterogenous maps

Shapeless provides a heterogenous map which supports an arbitrary relation between the key type and the corresponding value type,

edited Feb 04 '17 at 04:46

answered Feb 04 '17 at 00:44

rogue-one

11,259
7
53
75

Yes I want to avoid pattern matching. It comes with the same maintaining cost as `switch-case`. For new types I just want to register new handlers and mappers. – Feb 04 '17 at 08:35
@Sorona what about the HMap proposal will that work for you? – rogue-one Feb 04 '17 at 14:12
Never used `HMap` before and would prefer a pure Scala solution without additional dependency. – Feb 04 '17 at 17:48

thesamet · Answer 2 · 2017-02-06T00:48:22.640

One trick you can use is to capture the companion object as an implicit, and combine the parsing and handling in a single function where the type is available to the compiler:

case class Handler[T <: GeneratedMessage with Message[T]](handler: T => Unit)(implicit cmp: GeneratedMessageCompanion[T]) {
  def handle(bytes: ByteString): Unit = {
    val msg: T = cmp.parseFrom(bytes.newInputStream)
    handler(t)
  }
}

val handlers: Map[String, Handler[_]] = Map(
  "X" -> Handler((x: X) => Unit),
  "Y" -> Handler((x: Y) => Unit)
)

// To handle the request:
handlers(request.typeId).handle(request.message)

Also, take a look at any.proto which defines a structure very similar to your Message. It wouldn't solve your problem, but you can take advantage of it's pack and unpack methods.

Any is not yet available in Scala.js, but thanks, I will have a look — , Feb 05 '17 at 06:59

score 0 · Accepted Answer · answered Feb 05 '17 at 21:45

I settled for this solution for now (basically thesamet's, a bit adapted for my particular use-case)

trait Handler[T <: GeneratedMessage with Message[T], R]
{
    implicit val cmp: GeneratedMessageCompanion[T]
    def handle(bytes: ByteString): R = {
        val msg: T = cmp.parseFrom(bytes.newInput())
        handler(msg)
    }

    def apply(t: T): R
}

object Test extends Handler[Test, String]
{
    override def apply(t: Test): String = s"$t received and handled"

    override implicit val cmp: GeneratedMessageCompanion[Test] = Test.messageCompanion
}

Handling loads of different message-types at runtime in an elegant way

3 Answers3