ExecutionContext causes Akka dead letter

Question

For some reason I have to use gRPC and Akka at the same time. When this actor is started as a top actor, nothing goes wrong (in this little demo). But when it becomes a child actor, it cannot receive any messages, and the following is logged:

[default-akka.actor.default-dispatcher-6] [akka://default/user/Grpc] Message [AkkaMessage.package$GlobalStart] from Actor[akka://default/user/TrackerCore#-808631363] to Actor[akka://default/user/Grpc#-1834173068] was not delivered. [1] dead letters encountered.

The example core:

class GrpcActor() extends Actor {
    val ec = scala.concurrent.ExecutionContext.global
    val service = grpcService.bindService(new GrpcServerImpl(), ec)
    override def receive: Receive = {
        case GlobalStart() => {
            println("GlobalStart")
        }
        ...
    }
}

I tried to create a new ExecutionContext like:

scala.concurrent.ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))

Why is this happening, and how do can I debug a dead letters problem like this (no exception is thrown)?

Update:

Sorry I didn't list everything here. I used normal Main method to test GrpcActor as top actor, and ScalaTest to test it as child actor, which is a mistake.

class GrpcActorTest extends FlatSpec with Matchers{
    implicit val system = ActorSystem()
    val actor: ActorRef = system.actorOf(Props[GrpcActor])
    actor ! GlobalStart()
}

It is this empty test suite that active shutdown the whole actor system. But the problem is with this line

val service = grpcService.bindService(new GrpcServerImpl(), ec)

the delivery of GlobalStart() was delayed after the shutdown.

Without that line, message can be delivered before the shutdown.

Is this a normal behavior?

(My guess: it happened that the GlobalStart() was queued after the shutdown message with that line, which did some heavy work and made the difference in time)

Thank @chunjef for correcting my grammar mistakes and perfecting this question. — Skye347, Oct 15 '17 at 15:42

score 0 · Answer 1 · answered Oct 15 '17 at 08:23

0

Add a supervisor strategy to its parent, add println to the actor lifecycle. There is something that kill your actor . Finally, if you provide a complete example maybe I can say more :)

answered Oct 15 '17 at 08:23

Fabio Fumarola

430
3
9

score 0 · Accepted Answer · answered Oct 15 '17 at 21:50

One way to address the problem is to make service a lazy val:

class GrpcActor extends Actor {
  ...
  lazy val service = grpcService.bindService(new GrpcServerImpl(), ec)
  ...
}

A lazy val is useful for long-running operations: in this case, it defers the initialization of service until it is used for the first time. Without the lazy modifier, service is initialized when the actor is created.

An alternative approach is to add a Thread.sleep in your test to prevent the actor system from shutting down before the actor has fully initialized:

class GrpcActorTest extends FlatSpec with Matchers {
  ...
  actor ! GlobalStart()
  Thread.sleep(5000) // or whatever length of time is needed to initialize the actor
}

(As a side note, consider using the Akka Testkit for your actor tests.)

ExecutionContext causes Akka dead letter

2 Answers2