Backpressure observable based on resource scarcity

Question

In RxJava 1 / RxScala, how can I throttle/backpressure a source observable in the following situation?

def fast: Observable[Foo] // Supports backpressure

def afterExpensiveOp: Observable[Bar] = 
    fast.flatMap(foo => Observable.from(expensiveOp(foo))

// Signature and behavior is out of my control
def expensiveOp(foo: Foo)(implicit ec: ExecutionContext): Future[Bar] = {
   if(noResources()) Future.failed(new OutOfResourcesException())
   else Future { Bar() }
}

A possible solution is just blocking until. Which works,but that is very inelegant and prevents multiple simultaneous requests:

def afterExpensiveOp: Observable[Bar] = fast.flatMap(foo => 
   Observable.just(Observable.from(expensiveOp(foo)).toBlocking.head)
)

Probably you'll have to write your own operator for this. In RxScala, an operator is a function `Subscriber[T] => Subscriber[R]`, and you can apply it to an `Observable` using `lift`. At some point, the `Subscriber[R]` that you're creating will have to check if there are resources available, and if so, call its inherited `request` method to get more items from the `fast` observable. — Samuel Gruetter, Nov 09 '16 at 23:38

score 0 · Accepted Answer · answered Nov 15 '16 at 09:25

0

flatMap has a parameter to limit the number of concurrent subscribers. If you use this flatMap takes care of the backpressure for you.

def afterExpensiveOp = fast.flatMap(safeNumberOfConccurrentExpensiveOps, x => Observable.from(expensiveOp(x)))

answered Nov 15 '16 at 09:25

dtech

13,741
11
48
73

Backpressure observable based on resource scarcity

1 Answers1