How to throttle Futures with one-second delay with Akka

Question

I have list of URIs, each of which I want to request with a one-second delay in between. How can I do that?

val uris: List[String] = List()
// How to make these URIs resolve 1 second apart?
val responses: List[Future[Response]] = uris.map(httpRequest(_))

This looks like Java code, not Scala... you declare variables with `val` or `var` in Scala. — James Whiteley, Dec 18 '18 at 13:05

score 6 · Accepted Answer · answered Dec 18 '18 at 13:29

You could create an Akka Streams Source from the list of URIs, then throttle the conversion of each URI to a Future[Response]:

def httpRequest(uri: String): Future[Response] = ???

val uris: List[String] = ???

val responses: Future[Seq[Response]] =
  Source(uris)
    .throttle(1, 1 second)
    .mapAsync(parallelism = 1)(httpRequest)
    .runWith(Sink.seq[Response])

Dima · Answer 2 · 2018-12-18T13:12:41.290

Something like this perahps:

  @tailrec
  def withDelay(
    uris: Seq[String], 
    delay: Duration = 1 second, 
    result: List[Future[Response]] = Nil,
  ): Seq[Future[Response]] = uris match {
     case Seq() => result.reversed
     case (head, tail@_*) => 
        val v = result.headOption.getOrElse(Future.successful(null))
          .flatMap { _ => 
            akka.pattern.after(delay, context.system.scheduler)(httpRequest(head))
          }
        withDelay(tail, delay, v :: result)
   }

this has a delay before the first execution as well, but I hope, it's clear enough how to get rid of it if necessary ... Another caveat is that this assumes that all futures succeed. As soon as one fails, all subsequent processing is aborted. If you need a different behavior, you may want to replace the .flatMap with .transform or add a .recover etc.

You can also write the same with .foldLeft if preferred:

  uris.foldLeft(List.empty[Future[Response]]) { case (results, next) => 
    results.headOption.getOrElse(Future.successful(null))
      .flatMap { _ => 
        akka.pattern.after(delay, context.system.scheduler)(httpRequest(next))
      } :: results
  }.reversed

doesn't this delay every future by 1 second and not one after the other? like first is delayed 1 sec, second is delayed 2 seconds, third is delayed 3 seconds — eguneys, Dec 18 '18 at 13:16
It adds a 1 second delay before executing each future. So, effectively, yes, total of N seconds delay for the Nth execution. Isn't it what you wanted? — Dima, Dec 18 '18 at 17:58

score 0 · Answer 3 · edited Dec 18 '18 at 14:25

0

akka streams has it out of the box with the throttle function (taking into account that you are using akka-http and added tag for akka streams)

edited Dec 18 '18 at 14:25

James Whiteley

3,363
1
19
46

answered Dec 18 '18 at 13:17

Serhii Shynkarenko

716
3
10

How to throttle Futures with one-second delay with Akka

3 Answers3