Source.queue giving unexpected behavior

Question

It doesn't matter what arguments I supply for the bufferSize and overflowStrategy parameters of Source.queue, the result is always something like the output at the bottom. I was expecting to see the offer invocations and offer results complete more or less immediately, and to be able to see different processing and offer result messages based on bufferSize and overflowStrategy. What am I doing wrong here?

Code:

  def main(args: Array[String]): Unit = {
    implicit val system: ActorSystem = ActorSystem("scratch")
    implicit val materializer: ActorMaterializer = ActorMaterializer()
    implicit val executionContext: ExecutionContextExecutor = system.dispatcher

    val start = Instant.now()
    def elapsed = time.Duration.between(start, Instant.now()).toMillis
    val intSource = Source.queue[Int](2, OverflowStrategy.dropHead)
    val intSink = Sink foreach { ii: Int =>
      Thread.sleep(1000)
      println(s"processing $ii at $elapsed")
    }
    val intChannel = intSource.to(intSink).run()
    (1 to 4) map { ii =>
      println(s"offer invocation for $ii at $elapsed")
      (ii, intChannel.offer(ii))
    } foreach { intFutureOfferResultPair =>
      val (ii, futureOfferResult) = intFutureOfferResultPair
      futureOfferResult onComplete { offerResult =>
        println(s"offer result for $ii: $offerResult at $elapsed")
      }
    }
    intChannel.complete()

    intChannel.watchCompletion.onComplete { _ => system.terminate() }
  }

Output:

offer invocation for 1 at 72
offer invocation for 2 at 77
offer invocation for 3 at 77
offer invocation for 4 at 77
offer result for 1: Success(Enqueued) at 90
processing 1 at 1084
offer result for 2: Success(Enqueued) at 1084
processing 2 at 2084
offer result for 3: Success(Enqueued) at 2084
processing 3 at 3084
offer result for 4: Success(Enqueued) at 3084
processing 4 at 4084

score 0 · Accepted Answer · answered Jul 20 '18 at 20:53

I can get the expected behavior by replacing:

val intChannel = intSource.to(intSink).run()

with:

val (intChannel, futureDone) = intSource.async.toMat(intSink)(Keep.both).run()

and:

intChannel.watchCompletion.onComplete { _ => system.terminate() }

with:

futureDone.onComplete { _ => system.terminate() }

Fixed Code:

  def main(args: Array[String]): Unit = {
    implicit val system: ActorSystem = ActorSystem("scratch")
    implicit val materializer: ActorMaterializer = ActorMaterializer()
    implicit val executionContext: ExecutionContextExecutor = system.dispatcher

    val start = Instant.now()
    def elapsed = time.Duration.between(start, Instant.now()).toMillis
    val intSource = Source.queue[Int](2, OverflowStrategy.dropHead)
    val intSink = Sink foreach { ii: Int =>
      Thread.sleep(1000)
      println(s"processing $ii at $elapsed")
    }
    val (intChannel, futureDone) = intSource.async.toMat(intSink)(Keep.both).run()
    (1 to 4) map { ii =>
      println(s"offer invocation for $ii at $elapsed")
      (ii, intChannel.offer(ii))
    } foreach { intFutureOfferResultPair =>
      val (ii, futureOfferResult) = intFutureOfferResultPair
      futureOfferResult onComplete { offerResult =>
        println(s"offer result for $ii: $offerResult at $elapsed")
      }
    }
    intChannel.complete()

    futureDone.onComplete { _ => system.terminate() }
  }

Output

offer invocation for 1 at 84
offer invocation for 2 at 89
offer invocation for 3 at 89
offer invocation for 4 at 89
offer result for 3: Success(Enqueued) at 110
offer result for 4: Success(Enqueued) at 110
offer result for 1: Success(Enqueued) at 110
offer result for 2: Success(Enqueued) at 110
processing 3 at 1102
processing 4 at 2102

See also the accepted answer at https://stackoverflow.com/questions/41350715/backpressure-strategies-for-akka-stream-source-queue-not-working — SourceSimian, Jul 20 '18 at 21:03

Source.queue giving unexpected behavior

1 Answers1