Akka: is actor's mailbox recovered after actor system crash / restart?

Question

Im trying to implement Saga (Process Manager) with PersistentActor which updates multiple entities / aggregate roots to achieve (eventual) consistency without ACID transaction.

Let's say, in the middle of the process, between Process Manager received message and started processing it, jvm crashed.

Or application is being redeployed at this very moment.

Will Process Manager's mailbox be recovered after jvm (along with actor system) restart?

score 1 · Answer 1 · answered Jan 13 '18 at 15:26

1

The mailboxes are not recovered in case of JVM crash. It is the sender's responsability to ensure that its message was received/persisted. Between persistent actors, use At-Least-Once Delivery.

answered Jan 13 '18 at 15:26

Frederic A.

3,504
10
17

So does it mean that, if sender uses At-Least-Once Delivery and jvm crashed after sender sent the message and before receiver received it, this message will be re-sent after jvm/actor system restart? – Teimuraz Jan 13 '18 at 17:45
Short answer is yes, the slightly longer answer is that after a timeout, the sender will notice it didn't get the expected delivery confirmation and will resend. This also works if the sender crashes. – Frederic A. Jan 15 '18 at 04:12

score 1 · Answer 2 · answered Jan 14 '18 at 08:57

Given that you're already using PersistentActor, it's reasonable to leverage its AtLeastOnceDelivery functionality for what you need. AtLeastOnceDelivery essentially ensures that the persistent sender actor will keep sending messages to the receiver at a configurable frequency until it receives confirmation from the receiver (or from some termination routine for canceling ongoing delivery attempts).

As with general PersistentActor (with or without AtLeastOnceDelivery), messages that are persisted will be re-sent should the JVM crash. In the case of AtLeastOnceDelivery, persisted messages will be re-sent upon system recovery until receipt of confirmation. Akka provides method deliver to send each message tagged with a consecutive monotonically increasing deliveryId and, upon receiving a confirmation with the corresponding deliveryId from the receiver, uses method confirmDelivery to signal successful delivery of the message.

The following snippet, which is part of the sample code from relevant Akka doc, highlights the key AtLeastOnceDelivery logic in the sender actor class (which extends PersistentActor with AtLeastOnceDelivery). Note that the same persistence handler updateState is used to persist events corresponding to deliver and confirmDelivery for consistent state recovery:

override def receiveCommand: Receive = {
  case s: String =>
    persist(MsgSent(s))(updateState)
  case Confirm(deliveryId) =>
    persist(MsgConfirmed(deliveryId))(updateState)
}

override def receiveRecover: Receive = {
  case evt: Evt => updateState(evt)
}

def updateState(evt: Evt): Unit = evt match {
  case MsgSent(s) =>
    deliver(destination)(deliveryId => Msg(deliveryId, s))
  case MsgConfirmed(deliveryId) =>
    confirmDelivery(deliveryId)
}

Akka: is actor's mailbox recovered after actor system crash / restart?

2 Answers2