Akka/Scala: Can You Explain What's Going On in this Akka Streams Flow?

Question

I'm cutting my teeth on Akka streams and did a fibonacci publisher-subscriber example as follows. However, I don't quite understand yet how the demand is initially generated and what relation it has with the subscriber's request strategy. Can someone please explain?

FibonacciPublisher:

class FibonacciPublisher extends ActorPublisher[Long] with ActorLogging {
  private val queue = Queue[Long](0, 1)

  def receive = {
    case Request(_) => // _ is the demand
      log.debug("Received request; demand = {}.", totalDemand)
      publish
    case Cancel =>
      log.info("Stopping.")
      context.stop(self)
    case unknown => log.warning("Received unknown event: {}.", unknown)
  }

  final def publish = {
    while (isActive && totalDemand > 0) {
      val next = queue.head
      queue += (queue.dequeue + queue.head)

      log.debug("Producing fibonacci number: {}.", next)

      onNext(next)

      if (next > 5000) self ! Cancel
    }
  }
}

FibonacciSubscriber:

class FibonacciSubscriber extends ActorSubscriber with ActorLogging {
  val requestStrategy = WatermarkRequestStrategy(20)

  def receive = {
    case OnNext(fib: Long) =>
      log.debug("Received Fibonacci number: {}", fib)

      if (fib > 5000) self ! OnComplete
    case OnError(ex: Exception) =>
      log.error(ex, ex.getMessage)
      self ! OnComplete
    case OnComplete =>
      log.info("Fibonacci stream completed.")
      context.stop(self)
    case unknown => log.warning("Received unknown event: {}.", unknown)
  }
}

Fibonacci App:

val src = Source.actorPublisher(Props[FibonacciPublisher])
val flow = Flow[Long].map { _ * 2 }
val sink = Sink.actorSubscriber(Props[FibonacciSubscriber])

src.via(flow).runWith(sink)

Sample run: Question: Where did the initial demand for 4 come from?

2015-10-03 23:10:49.120 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Received request; demand = 4.
2015-10-03 23:10:49.120 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Producing fibonacci number: 0.
2015-10-03 23:10:49.121 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Producing fibonacci number: 1.
2015-10-03 23:10:49.121 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Producing fibonacci number: 1.
2015-10-03 23:10:49.121 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Producing fibonacci number: 2.
2015-10-03 23:10:49.122 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciSubscriber - Received Fibonacci number: 0
2015-10-03 23:10:49.122 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciSubscriber - Received Fibonacci number: 2
2015-10-03 23:10:49.123 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Received request; demand = 2.
2015-10-03 23:10:49.123 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciSubscriber - Received Fibonacci number: 2
2015-10-03 23:10:49.124 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciSubscriber - Received Fibonacci number: 4
2015-10-03 23:10:49.124 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Producing fibonacci number: 3.
2015-10-03 23:10:49.125 [fibonacci-akka.actor.default-dispatcher-2] [DEBUG] n.a.s.f.FibonacciProducer - Producing fibonacci number: 5.

score 1 · Answer 1 · answered Oct 05 '15 at 10:13

1

The initial demand to your source is provided by the input buffer of your later stages. This, in turn, is configured via the ActorMaterializerSettings instance that you pass when you initialise your ActorMaterializer.

If you don't pass any specific setting akka will use provided configuration to initialise one; in the default configuration you can find that akka.stream.materializer.initial-input-buffer-size is set to 4. Changing that should change your initial demand.

answered Oct 05 '15 at 10:13

Aldo Stracquadanio

6,167
1
23
34

Thanks for your answer. While playing with my fibonacci example above, instead of one subscriber, I attached 2, and the initial demand became 16. I see that 16 is the default value of `akka.stream.materializer.max-input-buffer-size`. I also read [Buffers and working with rate](http://doc.akka.io/docs/akka-stream-and-http-experimental/current/scala/stream-rate.html). What that doesn't explain, and I'm hoping you could, is that 1) when the demand is more than 4 but less than 16, what happens? And 2) what happens when the demand is more than 16? – Abhijit Sarkar Oct 06 '15 at 04:54
I think that in most stages the buffer will be used only when the source is quicker then the consumer; in that case filling the buffer (less demand) will result in back-pressure being applied to the consumer. If the consumer is quicker instead (more demand) the buffer won't be used at all and data from the producer will directly flow to the consumer. – Aldo Stracquadanio Oct 06 '15 at 18:52
What you said in your comment is incorrect. According to the doc [Buffers and working with rate](http://doc.akka.io/docs/akka-stream-and-http-experimental/current/scala/stream-rate.html), "for performance reasons Akka Streams introduces a buffer for every processing stage". – Abhijit Sarkar Oct 07 '15 at 05:12

Akka/Scala: Can You Explain What's Going On in this Akka Streams Flow?

1 Answers1