0

Any way out for sustaining a tail recursion once you go concurrent?

@tailrec // and we use an accumulator parameter for enabling that
def impl(
  apiCall:HttpRequest = initialApiCall, 
  soFarResult: List[JsValue] = List())
    : Future[List[JsValue]] = {

  nonBlockingHttp(apiCall) flatMap { response =>

      if (!response.isSuccess) 
        throw new Exception(s"""api call failed - have we been rate limited? failure details: \n$response""")

      val asJson: JsValue = Json.parse(response.body) //println(Json.prettyPrint(asJson))
      val headers = response.headers
      val linkHeaders = parseGithubLinkHeader(headers("Link"))
      println(linkHeaders)

      val projects = (asJson \ "items")
        .as[JsArray]
        .as[List[JsValue]]

      val result = soFarResult ++ projects 

      if (linkHeaders("next") == linkHeaders("last")) Future { result } 
      else impl(Http(linkHeaders("next")), result) 
    }
  }

Of course, this yields

could not optimize @tailrec annotated method impl: it contains a recursive call not in tail position

as the flatMap is the last call made, not the recursive call to impl at the tail of flatMap.

Robin Green
  • 32,079
  • 16
  • 104
  • 187
matanster
  • 15,072
  • 19
  • 88
  • 167
  • Tail-recursive means compiler can turn your recursive function into a loop. awaiting the result might do it, but it's probably not what you want. – Ashalynd Dec 30 '15 at 08:47
  • This code does not await the result. It (flat) maps it. – matanster Dec 30 '15 at 08:51
  • That is why it cannot be tail-recursive. – Ashalynd Dec 30 '15 at 08:54
  • @Ashalynd the question is asking for a way to be tail-recursive and concurrent, not to re-iterate that the provided code is not. These comments are hardly constructive as they are, and frankly, a waste of time. – matanster Dec 30 '15 at 08:56
  • I am trying to tell you that if you want your method to be tail-rec, you cannot use Futures, by definition of what tail-recursive means. – Ashalynd Dec 30 '15 at 09:02
  • But also you can't bust the stack by flatmapping futures, right? – Chris Martin Dec 30 '15 at 09:56
  • @ChrisMartin I thought so, but I tried to read the source code, and it's pretty unclear what it's doing, but it looks like it's running flatmaps immediately if the future has already completed. In that case, you *could* bust the stack by flatmapping futures, in theory. – Robin Green Dec 30 '15 at 11:03
  • 4
    This gets asked a lot. You can't make something that returns a `Future` tail recursive, because you need to `map` or `flatmap` the `Future` which will always put the recursive call outside the tail position. But you also don't need to make it tail recursive, because each call to a method that returns a `Future` has it's own stack, so you will not consume the whole stack. – Michael Zajac Dec 30 '15 at 13:59
  • @m-z I renamed the duplicate so that it would be searchable in the future, and hopefully this gets asked less. – matanster Dec 30 '15 at 14:31
  • Actually, here as well as in the duplicate, it sounds like the same stack space will be used, only it will be divided across threads. Not sure why this is really a solution by any order of magnitude, and it is alternatively near impossible to loop over a chain of `Future`s in scala without recursion or blocking. So I feel none of this is very great. – matanster Dec 30 '15 at 15:13

0 Answers0