RxJava Observable.fromEmitter odd backpressure behaviour

Question

I've been making use of Observable.fromEmitter() as a fantastic alternative to Observable.create(). I've recently run into some weird behaviour and I can't quite work out why this is the case. I'd really appreciate someone with some knowledge on backpressure and schedulers taking a look at this.

public final class EmitterTest {
  public static void main(String[] args) {
    Observable<Integer> obs = Observable.fromEmitter(emitter -> {
      for (int i = 1; i < 1000; i++) {
        if (i % 5 == 0) {
          sleep(300L);
        }

        emitter.onNext(i);
      }

      emitter.onCompleted();
    }, Emitter.BackpressureMode.LATEST);

    obs.subscribeOn(Schedulers.computation())
        .observeOn(Schedulers.computation())
        .subscribe(value -> System.out.println("Received " + value)); // Why does this get stuck at "Received 128"

    sleep(10000L);
  }

  private static void sleep(Long duration) {
    try {
      Thread.sleep(duration);
    } catch (InterruptedException e) {
      throw new RuntimeException(e);
    }
  }
}

The output of this application is

Received 1
Received 2
...
Received 128

Then it remains stuck at 128 (assumedly because this is RxJava's default buffer size).

If I change the mode specified in fromEmitter() to BackpressureMode.NONE, then the code works as intended. If I remove the call to observeOn(), it also works as intended. Is anyone able to explain why this is the case?

This is odd, it shouldn't stop at all. It stops even if using toBlocking() or a smaller buffer size in observeOn. I'll investigate this further. — akarnokd, Oct 20 '16 at 08:39
What is equivalent of Observable.fromEmitter in RX Java 2.0? — Mike6679, Apr 05 '17 at 16:16
`Observable.create()` replaces `fromEmitter()` in 2.0. If you want the old, scary create behaviour you use `Observable.unsafeCreate()`. — Chris Horner, Apr 07 '17 at 00:51

score 4 · Accepted Answer · answered Oct 20 '16 at 09:03

This is a same-pool deadlock situation. subscribeOn schedules the downstream request on the same thread it is using but if that thread is busy with a sleep/emission loop, the request gets never delivered to fromEmitter and thus after some time LATEST starts to drop elements up until the very end where the very last value (999) is delivered if the main source waits long enough. (This is a similar situation with onBackpressureBlock we removed.)

If subscribeOn didn't do this scheduling of requests, the example would work propery.

I've opened an issue to work out the solutions.

The workaround for now is to use bigger buffer size with observeOn (there's an overload) or use fromEmitter(f, NONE).subscribeOn().onBackpressureLatest().observeOn()

Thanks for the concise answer, David! Appreciate it. – Chris Horner Oct 20 '16 at 23:28 — Chris Horner, Oct 20 '16 at 23:28

score 2 · Answer 2 · edited May 23 '17 at 11:46

This is not odd, this is expected.

Let's trace the calls. Start with:

Observable.subscribe(Subscriber<? super T> subscriber)

Observable.subscribe(Subscriber<? super T> subscriber, Observable<T> observable)

RxJavaHooks.onObservableStart(observable, observable.onSubscribe).call(subscriber);

Subscriber<? super T> st = RxJavaHooks.onObservableLift(operator).call(o);

and so on. Look into the constructor of:

OperatorObserveOn(Scheduler scheduler, boolean delayError, int bufferSize):

public OperatorObserveOn(Scheduler scheduler, boolean delayError, int bufferSize) {
    this.scheduler = scheduler;
    this.delayError = delayError;
    this.bufferSize = (bufferSize > 0) ? bufferSize : RxRingBuffer.SIZE;
}

If you do not specify the buffer, the default is RxRingBuffer.SIZE, which size is platform dependent.

That is why, when you invoke observeOn operator without buffer size, the default is 128 (16 on Android).

The solution for this problem is very simple: just use another observeOn operator and declare buffer size. But if you declare buffer size for let's say 1000 (as many as elements coming from emitter), the program will still end without emitting all values (around 170). Why? Because program ends. The main Thread ends after 10 000 seconds and you calculation is done in another Thread (Schedulers.computation()). The solution for that? Use CountdownLatch. Be aware never use it production, it is helphul just for tests.

Thanks for the answer! I ultimately preferred David's answer, but I appreciate the effort you went to dissecting the calls. — Chris Horner, Oct 20 '16 at 23:30

RxJava Observable.fromEmitter odd backpressure behaviour

2 Answers2

Linked