Reactive (RX) throttle without loss

Question

Before I go write one I thought it would be worth asking: RX has a throttle extension method which discards events if they occur too quickly.

So if you asked it to throttle events to 1 per 5 seconds, if you received a event after 0.1second, then a second event 1 second later, you would get one event followed by silence.

What I want is for it to raise the first event after 0.1 seconds, but to then raise another event 4.9 seconds later.

Further, if I receive events at 0.1, 1 and 2 seconds, I want it to raise the event at 0.1 seconds, 5 seconds and then nothing, so I don't want it capturing n events and only releasing one per period for n periods.

Buffer does the opposite, in that it saves everything for 5 seconds and then raises the event, so what is neither throttle nor buffer, but something inbetween.

Is there a way to do this with the existing framework, or do I need to write one?

Can you please explain the second scenario? A(0.1s), B(1s), C(2s) => A(0.1s) and that is it? What is the difference from the first scenario? — Tamas Ionut, Mar 14 '16 at 12:47
Sure Tamas. A(0.1s),B(1s), C(2s) would result in A(0.1s), B(5s). If we just rate limited it you might end up with A(0.1s), B(5s), C(10s), which I do not want. So you're correct the result is the same, but the input is not. Does that make more sense? — Ian, Mar 14 '16 at 19:24
So, regardless of how many events you receive in 5, you only want the first two, first at the exact order in which it appeared and the second at the edge of the buffer interval? — Tamas Ionut, Mar 14 '16 at 19:55
Yes Tamas, imagine you have a bus and a database. Someone writes a record to the database and then sends a message on the bus which a service picks up. It then goes to the database to get the record. Now imagine a second record is written and second message is sent. If we're using Throttle, RX would ignore the second message, so we wouldn't check the database again unless a new request comes in after. If we use Buffer there is a delay of 5 seconds before starting processing. — Ian, Mar 15 '16 at 13:10

score 6 · Accepted Answer · answered Mar 15 '16 at 08:33

I think you will have to write your own operator, or do some toying around with Window. Like the other comments, I am not 100% sure on your requirements, but I have tried to capture them in these tests.

using System;
using System.Reactive.Linq;
using Microsoft.Reactive.Testing;
using NUnit.Framework;

[TestFixture]
public class Throttle : ReactiveTest
{
    private TestScheduler _testScheduler;
    private ITestableObservable<int> _sourceSequence;
    private ITestableObserver<int> _observer;

    [SetUp]
    public void SetUp()
    {
        var windowPeriod = TimeSpan.FromSeconds(5);
        _testScheduler = new TestScheduler();
        _sourceSequence = _testScheduler.CreateColdObservable(
            //Question does the window start when the event starts, or at time 0?
            OnNext(0.1.Seconds(), 1),
            OnNext(1.0.Seconds(), 2),
            OnNext(2.0.Seconds(), 3),
            OnNext(7.0.Seconds(), 4),
            OnCompleted<int>(100.0.Seconds())
            );

        _observer = _testScheduler.CreateObserver<int>();
        _sourceSequence
            .Window(windowPeriod, _testScheduler)
            .SelectMany(window =>
                window.Publish(
                    shared => shared.Take(1).Concat(shared.Skip(1).TakeLast(1))
                )
            )
            .Subscribe(_observer);
        _testScheduler.Start();
    }

    [Test]
    public void Should_eagerly_publish_new_events()
    {
        Assert.AreEqual(OnNext(0.1.Seconds(), 1), _observer.Messages[0]);
    }

    [Test]
    public void Should_publish_last_event_of_a_window()
    {
        //OnNext(1.0.Seconds(), 2) is ignored. As OnNext(5.0.Seconds(), 3) occurs after it, and before the end of a window, it is yeiled.
        Assert.AreEqual(OnNext(5.0.Seconds(), 3), _observer.Messages[1]);
    }

    [Test]
    public void Should_only_publish_event_once_if_it_is_the_only_event_for_the_window()
    {
        Assert.AreEqual(OnNext(7.0.Seconds(), 4), _observer.Messages[2]);
        Assert.AreEqual(OnCompleted<int>(100.0.Seconds()), _observer.Messages[3]);
    }

    [Test]
    public void AsOneTest()
    {
        var expected = new[]
        {
            OnNext(0.1.Seconds(), 1),
            //OnNext(1.0.Seconds(), 2) is ignored. As OnNext(5.0.Seconds(), 3) occurs after it, and before the end of a window, it is yeiled.
            OnNext(5.0.Seconds(), 3),
            OnNext(7.0.Seconds(), 4),
            OnCompleted<int>(100.0.Seconds())
        };
        CollectionAssert.AreEqual(expected, _observer.Messages);
    }
}

I think you're absolutely right, it's going to take a little work. What is extra useful is I wasn't aware of the Microsoft ReactiveTest class before seeing this code. I've had to do "things" to test before and this looks much cleaner. Thank you. — Ian, Mar 15 '16 at 13:13
NP. If you come up with a solution, it would be great if you posted it here with your tests too. — Lee Campbell, Mar 16 '16 at 01:22
I'm going to prod at it this weekend. I'd like an RX solution as I'm trying to drive adoption here and this would make a nice show case. — Ian, Mar 17 '16 at 14:49
Sounds good, I strongly urge youto come up with a suite of tests first so that you know you have considered all of the nominal scenarios & edge cases (No items in a window, single item in a window, two items in a window, many items in a window, consecutive empty windows, onError, onComplete etc...) — Lee Campbell, Mar 17 '16 at 23:36
I think that's great advice. I had no idea about the test support library I've just installed it from nuget, so I can have a play and figure out how that all works first. — Ian, Mar 18 '16 at 17:15

Reactive (RX) throttle without loss

1 Answers1