Rx .NET - force buffer to emit

Question

I want to use Rx Buffer functionality:

var source = new Subject<Price>();
var buffer = source
    .Buffer(TimeSpan.FromSeconds(30), 5)
    .Where(p => p.Any());

that means emit (publishing to subscribers) happens when buffer reaches size of 5 or 30 seconds have gone since the last emit.

But I need to be able to emit on demand - for example when I receive high priority sequence item. Then I want to add it to observable (source.OnNext()) and somehow force it to emit (that means returning all elements in the buffer and clearing it).

I know that I can add following code:

var flusher = new Subject<Price>();
var closing = flusher.Select(x => new List<Price> {x});
var query = buffer.Merge(closing).Subscribe(something);

and invoke flusher.OnNext(highPriorityItem) and I will have it emitted.

But in this case, I have two independent sequences with two different emits. I need one emit when buffer is full or specific item appears in sequence.

Force flush count-type Observable.Buffer c# and Force flush to Observable.Buffer c# don't seem to be suitable for me

`[...] don't seem to be suitable for me` Any particular reason? The latter seems to do exactly what you're asking for... — decPL, Oct 11 '17 at 12:44
There are some Unit-related definitions and Timer instead of Buffer. I don't know/understand how they fix my problem — brolly87, Oct 11 '17 at 12:54
Have you tried implementing those? Just replace that `Unit` with your `Price` class. If you have a specific problem with them, describe it and hopefully someone can help. — decPL, Oct 11 '17 at 13:04
When using Amb, just like in https://stackoverflow.com/questions/12944716/force-flush-to-observable-buffer-c-sharp?noredirect=1&lq=1example, I flush the buffer of prices when I use flusher.OnNext(somePrice). But I receive collection without somePrice object and additionally without doing that my buffer never flushes (and it should when it reaches size of 5) — brolly87, Oct 11 '17 at 13:15

Enigmativity · Answer 1 · 2017-10-12T10:08:55.513

I think decPL has the basic idea right here, but his solution isn't stable. Depending on the scheduler of the input observable you can get unpredictable results even if it's subscribed in the right order. That's because there are multiple independent subscriptions to input. You need to push this all through a .Publish(...) call to ensure only one subscription.

Also it need a way of cleaning up when the subscription is disposed. So it also needs to run through a .Create(...) call.

Here's how:

var input = new Subject<Price>();

IObservable<IList<Price>> query =
    input
        .Publish(i =>
            Observable
                .Create<IList<Price>>(o =>
                {
                    var timeBuffer =
                        Observable
                            .Timer(TimeSpan.FromSeconds(10.0))
                            .Select(n => Unit.Default);
                    var flush =
                        i
                            .Where(p => p.IS_IMPORTANT)
                            .Select(n => Unit.Default);
                    var sizeBuffer =
                        i
                            .Buffer(5)
                            .Select(l => Unit.Default);
                    return
                        i
                            .Window(() => Observable.Merge(timeBuffer, sizeBuffer, flush))
                            .SelectMany(w => w.ToList())
                            .Subscribe(o);
                }));

query.Subscribe(w => DO_SOMETHING_WITH_PRICES(w));

If `IS_IMPORTANT` is the only case for special "flushing", could you also replace the `flush` Subject as an Observable: `i.Where(p => p.IS_IMPORTANT).Select(_ => Unit.Default )` ) ? — supertopi, Oct 12 '17 at 08:15

decPL · Answer 2 · 2017-10-12T06:51:20.537

3

EDIT: @Enigmativity is absolutely correct, refer to his answer. Leaving this one intact, as hopefully it's a bit easier to determine the thought process here.

Try something as follows:

var input = new Subject<Price>(); //your input observable

var flush = new Subject<long>(); //used to manually flush the 'buffer' for important prices
var timeBuffer
   = Observable.Timer(TimeSpan.FromSeconds(10)); //controls the time-based part of 'buffer'
var sizeBuffer = input.Buffer(5).Select(l => 0L); //controls the size-based part of 'buffer'

var bufferedInput = input.Window(()=>Observable.Merge(timeBuffer, sizeBuffer, flush))
                         .SelectMany(w => w.ToList())
                         .Subscribe(w => DO_SOMETHING_WITH_PRICES(w));

//Flush on important price (NOTE - order of the two subscriptions matter)
input.Where(p => p.IS_IMPORTANT).Subscribe(p => flush.OnNext(0L));

edited Oct 12 '17 at 06:51

answered Oct 11 '17 at 14:05

decPL

5,384
1
26
36

This is almost ok, but when I push important price via OnNext, all of the elements in the buffer get emited, but this important price doesn't. It will get emited in the next round (when buffer gets size of 5 or after 30 seconds).I would like it to be a part of a first emit – brolly87 Oct 11 '17 at 14:24
Which is why I wrote that the order of subscriptions matter. Make sure you subscribe to the 'buffer' before the subscription forwarding important prices to `flush`. – decPL Oct 11 '17 at 14:47
1

Thanks, ignored that initially. Works now ! – brolly87 Oct 11 '17 at 14:51
2

This is an unstable solution - depending on the scheduler of the `input` observable you can get unpredictable results even if it's done in the right order. That's because there are multiple independent subscriptions to `input`. You need to push this all through a `.Publish()` call to ensure only one subscription. – Enigmativity Oct 11 '17 at 23:16

Rx .NET - force buffer to emit

2 Answers2