Creating an actorpublisher and actorsubscriber with same actor

Question

I'm a newbie to akka streams. I'm using kafka as a source(using ReactiveKafka library) and do some processing of data through the flow and using a subscriber(EsHandler) as sink.

Now I need to handle errors and push it to different kafka queue through an error handler. I'm trying to use EsHandler both as publisher and subscriber. I'm not sure how to include EsHandler as an middle man instead of sink.

This is my code:

val publisher = Kafka.kafka.consume(topic, "es", new StringDecoder())

val flow = Flow[String].map { elem => JsonConverter.convert(elem.toString()) }

val sink = Sink.actorSubscriber[GenModel](Props(classOf[EsHandler]))

Source(publisher).via(flow).to(sink).run()


class EsHandler extends ActorSubscriber with ActorPublisher[Model] {

  val requestStrategy = WatermarkRequestStrategy(100)

  def receive = {
    case OnNext(msg: Model) =>
      context.actorOf(Props(classOf[EsStorage], self)) ! msg

    case OnError(err: Exception) =>
      context.stop(self)

    case OnComplete =>
      context.stop(self)

    case Response(msg) =>
      if (msg.isError()) onNext(msg.getContent())
  }
}

class ErrorHandler extends ActorSubscriber {

  val requestStrategy = WatermarkRequestStrategy(100)

  def receive = {
    case OnNext(msg: Model) =>
      println(msg)
  }
}

score 3 · Answer 1 · answered Jul 07 '15 at 15:19

3

We highly recommend against implementing your own Processor (which is the name the reactive streams spec gives to "Subscriber && Publisher". It is pretty hard to get it right, which is why there's not Publisher exposed directly as helper trait.

Instead, most of the time you'll want to use Sources/Sinks (or Publishers/Subscribers) provided to you and run your operations between those, as map/filter etc. steps.

In fact, there is an existing implementation for Kafka Sources and Sinks you can use, it's called reactive-kafka and is verified by the Reactive Streams TCK, so you can trust it to be valid implementations.

answered Jul 07 '15 at 15:19

Konrad 'ktoso' Malawski

13,102
3
47
52

I'm using reactive kafka's publisher as a source. There is an actor in the middle of my flow. I need to handle the error messages and push them back to kafka. which is the preferred way to do? – Yoda Jul 07 '15 at 15:27
Also, I'll be using ReactiveKafka's subscriber(publisher to kafka) as a sink. – Yoda Jul 07 '15 at 15:39
2

`source.mapAsync(m => myActor ? m).to(sink).run()` – Konrad 'ktoso' Malawski Jul 07 '15 at 15:43
And, if I want to limit the incoming data(in case of temporary server failure) from my source, I wont be able to do that with ask pattern. That is the reason I moved to my own processor. Is there any way to achieve the same? – Yoda Jul 07 '15 at 15:49
@Konrad'ktoso'Malawski Could you please take a look at related question ? http://stackoverflow.com/q/32290285/226895 – expert Aug 29 '15 at 21:05

Creating an actorpublisher and actorsubscriber with same actor

1 Answers1