How can I block terminating my program until the Observable consumption is complete?

Question

I am currently trying to use Monix for throttling api get requests. I have tried using STTP's Monix backend, and it worked fine until couldn't shut down the Monix backend after I was done... As this seems more like an sttp issue than a Monix one, I tried to re-approach the problem by using sttp's default backend, while still using Monix to throttle.

I am mainly struggling with closing the monix backend once I am done with consuming the observable

I have tried to simplify the problem through:

  import monix.execution.Scheduler.Implicits.global

  val someIter = List(Task(1), Task(2))

  val obs: Observable[CancelableFuture[Int]] = Observable
    .fromIterable(someIter)
    .throttle(3.second, 1)
    .map(_.runToFuture)

However, I am still not sure how to turn off the program after the Observable is consumed, as it terminates prematurely here (unlike the monix backend case)...

In other words, how can I block terminating program until the Observable iterable is complete?

You probably want to create your client as a **Resource** then consume that resource into an **Observable** and `flatMap` it to use the client to create the observable of the requests. - In any case, you will be very lost with all this if you first do not understand what is the point of effects and how you compose them into your whole program. For example, you shouldn't be calling `_.runToFuture` - Also, you may want to give it a look to the docs of **monix**, you will see that observable exposes many combinators for this kind of scenarios, like `doOnComplete`. — Luis Miguel Mejía Suárez, Aug 10 '20 at 16:55

score 3 · Accepted Answer · answered Aug 10 '20 at 20:16

3

You can create Promise, complete it when Observable is completed by .doOnComplete

And await it in the main thread.

import monix.execution.Scheduler.Implicits.global

val someIter = List(Task(1), Task(2))
val promise = Promise()
val obs: Observable[CancelableFuture[Int]] = Observable.fromIterable(someIter).throttle(3.second, 1)
  .map(_.runToFuture)
  .doOnComplete(Task { promise.complete(Success()) })

Await.ready(promise.future, Duration.Inf)

answered Aug 10 '20 at 20:16

Artem Sokolov

810
4
8

Hi Artem : ) what kind of value would I need to wrap in the `Success()`? – alt-f4 Aug 12 '20 at 19:44

score 1 · Answer 2 · answered Aug 12 '20 at 19:42

In addition to the accepted answer by Artem, and with insights from the Monix Gitter community, another potential implementation could be:

  val someIter = List(Task(1), Task(2))
  val obs =
    Observable
      .fromIterable(someIter)
      .throttle(1 second, 10)
      .mapParallelUnordered(10)(x => x.map(x => x.send().body)) // Here we send requests
      .sumL // Sum just as an example 
      .runSyncUnsafe()

How can I block terminating my program until the Observable consumption is complete?

2 Answers2