0

I am trying to use the Amqp alpakka connector as a source and sink.

Source<CommittableReadResult, NotUsed> AMQP_SOURCE  
      -> processAndGetResponse 
      -> Sink<ByteString, CompletionStage<Done>> AMQP_SINK

I want to acknowledge the message obtained from the Amqp queue, after the Sink's operation is successful. like this:

amqp_source(committableReadResult) 
    -> processAndGetResponse 
    -> amqp_sink 
    -> IfSinkOperationSuccess.Then(committableReadResult.ack())

How can I achieve this? I basically want to mark the message as acknowledge only after the sink operation is successful.

Jerald Baker
  • 1,121
  • 1
  • 12
  • 48
  • If you using akka-stream `v2.5.20+`, I suggest to look at `SourceWithContext` and [Software Mill blog](https://blog.softwaremill.com/painlessly-passing-message-context-through-akka-streams-1615b11efc2c) – chauhraj Mar 22 '20 at 04:32

1 Answers1

0

In general, if you have a Sink which materializes into a Future[Done] (Scala API, I believe the Java equivalent is CompletionStage[Done]), you can just run a substream with that Sink within a mapAsync stage. Data will only pass through when successful.

From a quick perusal of the Alpakka AMQP API, the Sinks all materialize that way, so that approach will work.

Assuming that your doSomethingWithMessage function results in both the CommittableReadResult and a WriteMessage, something like the following in Scala will work:

def doSomethingWithMessage(in: CommittableReadResult): (CommittableReadResult, WriteMessage) = ???

amqpSource
  .map(doSomethingWithMessage)
  .mapAsync(parallelism) { tup =>
    val (crr, wm) = tup
    val fut = Source.single(crr, wm).runWith(amqpSink)
    fut.flatMap(_ => crr.ack().map(_ => crr.message))
  }.runWith(Sink.ignore)
Levi Ramsey
  • 18,884
  • 1
  • 16
  • 30
  • Thanks for the response. This is helpful. In this case, won't the sink be kinda reinitialized a.k.a create a new connection to the amqp server for every element in the stream? – Jerald Baker Mar 18 '20 at 11:33