2

I would like to write an Rx query that takes an IObvservable<char> and produces an IObservable<string>. The strings should be buffered until there have been no characters produced for a specified time.

The data source is a serial port from which I have captured the DataReceived event and from that I produce an IObservable<char>. The protocol I am dealing with is fundamentally character based, but it is not very consistent in its implementation so I need to observe the character stream in various different ways. In some cases there is an end-of-response terminator (but not a newline) and in one case, I get a string of unknown length and the only way I know it has all arrived is that nothing else arrives for a few hundred milliseconds. That is the problem I am trying to solve.

I have discovered

var result = source.Buffer(TimeSpan.FromMilliseconds(200))
                    .Select(s=>new string(s.ToArray()));

Buffer(TimeSpan) is almost what I need but not quite. I need the timer to reset every time a new character arrives, so that the buffer is only produced when sufficient time has elapsed since the last character.

Please, can anyone offer a suggestion on how to achieve this?

[Update] While I was waiting for an answer, I came up with a solution of my own which essentially re-invents Throttle:

    public virtual IObservable<string> BufferUntilQuiescentFor(IObservable<char> source, TimeSpan quietTime)
    {
        var shared = source.Publish().RefCount();
        var timer = new Timer(quietTime.TotalMilliseconds);
        var bufferCloser = new Subject<Unit>();
        // Hook up the timer's Elapsed event so that it notifies the bufferCloser sequence
        timer.Elapsed += (sender, args) =>
        {
            timer.Stop();
            bufferCloser.OnNext(Unit.Default);  // close the buffer
        };
        // Whenever the shared source sequence produces a value, reset the timer, which will prevent the buffer from closing.
        shared.Subscribe(value =>
        {
            timer.Stop();
            timer.Start();
        });
        // Finally, return the buffered sequence projected into IObservable<string>
        var sequence = shared.Buffer(() => bufferCloser).Select(s=>new string(s.ToArray()));
        return sequence;
    }

I wasn't understanding Throttle correctly, I thought it behaved differently than it actually does - now that I've had it explained to me with a 'marble diagram' and I understand it correctly, I believe it is actually a much more elegant solution that what I came up with (I haven't tested my code yet, either). It was an interesting exercise though ;-)

Tim Long
  • 13,508
  • 19
  • 79
  • 147
  • I have solved 'wait for inactivity' by setting a timer to fire after the inactive period but resetting it back to start fresh again on any activity. Then flushing the buffer on timer expire events – PunkUnicorn May 19 '15 at 22:16

2 Answers2

4

Does this do what you need?

var result =
    source
        .Publish(hot =>
            hot.Buffer(() =>
                hot.Throttle(TimeSpan.FromMilliseconds(200))))
       .Select(s => new string(s.ToArray()));
Enigmativity
  • 113,464
  • 11
  • 89
  • 172
  • Now I'm stuck, I want to accept both answers but I can only accept one. It is a difficult decision but I'm accepting the other answer because he took time to explain to me how it works, which is what I really needed - the understanding rather then the code snippet. But many, many thanks for inspiring the correct answer. – Tim Long May 19 '15 at 23:35
  • I sincerely was not trying to hijack your answer and hope there are no hard feelings. The explanation was too long for a comment (and no marble diagrams!). – Timothy Shields May 20 '15 at 01:55
  • 1
    @TimothyShields - No, that's fine. You provided an explanation that my answer really did need. Good job. – Enigmativity May 20 '15 at 02:04
4

All credit for this goes to Enigmativity - I'm just repeating it here to go with the explanation I'm adding.

var dueTime = TimeSpan.FromMilliseconds(200);
var result = source
    .Publish(o => o.Buffer(() => o.Throttle(dueTime)))
    .Select(cs => new string(cs.ToArray()));

The way it works is shown in this figure (where dueTime corresponds to three dashes of time):

source:    -----h--el--l--o----wo-r--l-d---|
throttled: ------------------o------------d|
buffer[0]: -----h--el--l--o--|
buffer[1]:                    -wo-r--l-d--|
result:    ------------------"hello"------"world"

The use of Publish is just to make sure that Buffer and Throttle share a single subscription to the underlying source. From the documentation for Throttle:

Ignores the values from an observable sequence which are followed by another value before due time...

The overload of Buffer being used takes a sequence of "buffer closings." Each time the sequence emits a value, the current buffer is ended and the next is started.

Timothy Shields
  • 75,459
  • 18
  • 120
  • 173
  • I'm starting to get the hang of this now... my app is talking to my device. It takes quite a paradigm shift to start thinking about things as sequences of data in motion - but I'm really liking how it works so far and how easy LINQ makes it to get just the data you want. – Tim Long May 20 '15 at 11:12
  • @TimLong Think about how difficult it would be to get the exact same behavior as this query without using Rx. Then look at how you can do it in two lines with Rx. It's an extremely powerful tool. – Timothy Shields May 20 '15 at 14:59