Akka HTTP flow not stopping when exception thrown

Question

I'm trying to use Akka HTTP for POSTing to a webserver. If a POST fails I would like it to stop and not send more POSTs as they are not idempotent.

The code below creates POSTs and sends them to a test web server. It throws an exception on the first response. The code should be runnable in which case you'll see it prints:

i = 0
got response
i = 1
stopping
Exception in thread "main" java.lang.Exception
i = 2
i = 3
i = 4
i = 5

So the 'stopping' happens after the next request has been put together (i = 1), then the code just continues.

Does anyone know how to stop the flow once there's an error and to not send any further POSTs?

(Scala 2.11.8, Akka 2.4.4)

object FlowTest {
  def main(args: Array[String]) {
    val stop: Supervision.Decider = {
      case _ =>
        println("stopping")
        Supervision.Stop
    }

    implicit val system = ActorSystem()
    import system.dispatcher
    implicit val mat = ActorMaterializer()
    val connectionFlow: Flow[HttpRequest, HttpResponse, Future[Http.OutgoingConnection]] =
      Http().outgoingConnection(host = "posttestserver.com", port = 80)

    val future: Future[Done] = Source(0 to 10).map {
      i =>
        val uri = s"/post.php?dir=so_akka&i=$i"
        println(s"i = $i")
        HttpRequest(method = HttpMethods.POST, uri = uri, entity = s"data $i")
    }.via(connectionFlow).mapAsync(1) {
      resp =>
        Unmarshal(resp.entity).to[String]
          .map { str =>
            println(str)
            throw new Exception("") // Always fail
            str
          }
    }.withAttributes(ActorAttributes.supervisionStrategy(stop)).runForeach(println)

    Await.result(future, Duration.Inf)
  }
}

Since the stream is processed asynchronously, I don't think it is possible to cancel the stream based on a condition (e.g. an exception). It might be possible that the results of the completed futures of the subsequent elements have already been emitted downstream. If you really need to cancel after an exception is thrown, you would need to make sure that the elements are processed sequentially, probably by blocking the futures. — devkat, May 10 '16 at 07:41

score 0 · Answer 1 · answered May 10 '16 at 10:31

So I think there were two problems I was having with the above code.

HTTP POSTs shouldn't be pipelined. I was hoping Akka HTTP would wait until one POST was processed, with no errors, before sending the next. This doesn't happen.
Exceptions were not being propagated up the flow. So throwing in the processing code didn't stop the Source from creating more POSTs and them being sent.

So there are two fixes.

I have set the withSyncProcessingLimit on the ActorMaterializer to one. Which stops the Source from sending new messages before they have been processed. I've also had to change the .mapAsync part so there is now a .map which checks the status code and errors if needed, and a .mapAsync which looks at the response body. You can't look at the response body in the .map part.
I've added a KillSwitch to stop the flow. Throwing an exception should have the same effect but doesn't. So this is a horrible hack but works.

I think there must be a better way of doing this. Using an Akka HTTP flow with HTTP POSTs shouldn't be so painful.

Here is the new code.

object FlowTest {
  def main(args: Array[String]) {
    implicit val system = ActorSystem()
    import system.dispatcher
    implicit val mat = ActorMaterializer.create(
      ActorMaterializerSettings.create(system).withSyncProcessingLimit(1), system
    )
    val connectionFlow = Http().outgoingConnection(host = "posttestserver.com", port = 80)
    val source = Source(0 to 10)
    val killSwitch = KillSwitches.shared("HttpPostKillSwitch")

    try {
      val future: Future[Done] = source.via(killSwitch.flow).map {
        i =>
          val uri = s"/post.php?dir=test&i=$i"
          println(s"i? = $i")
          HttpRequest(method = HttpMethods.POST, uri = uri, entity = s"data $i")
      }
        .via(connectionFlow)
        .map {
          resp =>
            println("got response")
//          if(resp.status != OK) { // always fail for testing
              val e = new Exception("")
              killSwitch.abort(e)
              throw e
//          }
            resp
        }
        .mapAsync(1) {
          resp =>
            Unmarshal(resp.entity).to[String]
              .map { str =>
                println("got " + str)
                str
              }
        }
        .runForeach(println)

      Await.result(future, Duration.Inf)
    } catch {
      case NonFatal(e) =>
        system.terminate()
    }
  }
}

Akka HTTP flow not stopping when exception thrown

1 Answers1