RxJava onBackpressureBuffer not emitting items

Question

I've witnessed a weird behavior with onBackpressureBuffer, I'm not sure if it is a valid behavior or a bug.

I'm having a tcp call that is emitting items in a certain rate (using streaming and inputStream but that just for some info)

On top of it I've created an observable using create that will emit an item each time it is ready.

Let's call it messages().

Then I'm doing this:

messages()
   .subscribeOn(Schedulers.io())
   .observeOn(AndroidSchedulers.mainThread())
   .subscribe({//do some work});

I've noticed using analytics tools that MissingBackPressureException is thrown rarely, so I've added onBackpressureBuffer to the call.

If I'm adding it after observeOn:

messages()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .onBackpressureBuffer()
    .subscribe({//do some work})

everyting works fine, but it means it will buffer only after it get's to the UI Main thread, so I prefered it to be like this:

messages()
    .onBackpressureBuffer()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe({//do some work});

And that where things start to get weird.

I've noticed the while messages() keeps emitting item, at some point they will stop being delivered to the subscriber.

More precisely after exactly 16 items, what is apparently happing is that the buffer will start holding items without passing them forward.

Once I cancel the messages() with some sort of timeout mechanism, it will cause messages() to emit onError() and the buffer will emit immediately all the items it has kept (they will be handled).

I've checked to see if it is the subscriber fault of doing too much work but it not, he is finished and still he doesn't get the items...

I've also tried using the request(n) method in the subscriber asking for one item after onNext() is finished but the buffer doesn't behave.

I suspect that the messaging system of the Main Android UI Thread with the backpressure causing this, but I can't explain why.

can someone explain why this is happening? is this a bug or a valid behaviour? Tnx!

score 5 · Accepted Answer · edited May 23 '17 at 12:31

5

Not knowing how messages(), based on the behavior described, this is a similar same-pool deadlock as with this question

The workaround, what you didn't try, is to put the .onBackpressureBuffer between the subscribeOn and observeOn.

messages()
.subscribeOn(Schedulers.io())
.onBackpressureBuffer()           // <---------------------
.observeOn(AndroidSchedulers.mainThread())
.subscribe({//do some work});

edited May 23 '17 at 12:31

Community

1
1

answered Oct 20 '16 at 09:12

akarnokd

69,132
14
157
192

interesting, I'll check it out – ndori Oct 20 '16 at 09:23
messages() is using inputStream from responseBody.byteStream() of okHttp3, it then uses a while loop on the inputStream with .read(). everytime the is a valid message it will emit it and continue reading. I'm using a delimiter to distinguish between the start and the end of a message. if I get an IOException from the read() I will call onError. – ndori Oct 20 '16 at 09:25
that worked perfectly, I didn't know there is a difference between putting a call after and not before of subsribeOn(), I'm usually using the pattern of putting subsribeOn(io()).observeOn(ui()) lastly in the chain just before subsribe, is that wrong? I will love a pointing direction to understand this area more thoroughly. – ndori Oct 20 '16 at 09:49

score 1 · Answer 2 · edited May 23 '17 at 11:53

Question is different but answer comes down to the same thing. The implementation of observeOn constructor: OperatorObserveOn(Scheduler scheduler, boolean delayError, int bufferSize):

public OperatorObserveOn(Scheduler scheduler, boolean delayError, int bufferSize) {
    this.scheduler = scheduler;
    this.delayError = delayError;
    this.bufferSize = (bufferSize > 0) ? bufferSize : RxRingBuffer.SIZE;

}

The last line points to the buffer size.

The buffer size on Android is 16.

The solution is simply passing a bigger buffer size to observeOn(Scheduler scheduler, int bufferSize) operator:

messages()
    .observeOn(AndroidSchedulers.mainThread(), {buffer_size})

Be careful not to put too high value, as Android has limited memory.

tnx, While this explains the "magic" of 16, it is only the symptom but not the actual problem. @akarnokd does solves this exactly the way it is intended. — ndori, Oct 20 '16 at 09:50

RxJava onBackpressureBuffer not emitting items

2 Answers2