Drop any except last sample in observable sequence until consumer is ready

Question

There is a real-time sequence of instrumental observations IObservable<S>. There is also an async consumer function f : S -> Task<U> (with side effects) that may take time to process a value. The function must not be reentered. As soon as it has returned, however, the next available sample from the stream should be passed to it. Additionally, the function must see the very last sample at the end of the stream.

It feels like this must be a pretty common pattern, but I cannot find an idiomatic solution. My approach works, but rather reeks of inelegance of using a subject as an imperative signal variable in the feedback loop. Is there an idiomatic solution to my problem?

static void Main() {
  var source = Observable.Interval(TimeSpan.FromMilliseconds(100)).Take(27);

  var s = source.Publish(ps => {
    var signal = new Subject<Unit>();
    return ps
      .SkipLast(1)
      .Window(signal)
      .SelectMany(w => w.Take(1))
      .Merge(ps.PublishLast().RefCount())
      .Select(x => Observable.FromAsync(() => LengthyWork(x)))
      .Concat()
      .Do(signal.OnNext);
  });
  var d = s.Subscribe();
  Thread.Sleep(5000);
  Console.WriteLine("Stopping");
  d.Dispose();
}

static async Task<Unit> LengthyWork(long n) {
  Console.WriteLine($"Processing {n}");
  await Task.Delay(800);
  return Unit.Default;
}

EDIT: The code actually does not solve the problem in question: .Merge() composition with the published last element pushes that element out of sync. In this example terms, 26 is send before the function returns from processing 24.

Output (note that item 26 is the last before end of stream).

Processing 0
Processing 8
Processing 16
Processing 24
Processing 26
Stopping

You say "The function must not be reentered. As soon as it has returned, however, the next available sample from the stream should be passed to it." - That's how Rx works by default. — Enigmativity, May 15 '16 at 11:08
@Enigmativity: Indeed it is, except this is an `async` function. The thread returns from it (and thus from `Observer.OnNext`) when the function yields, like when execution hits an `await` inside it. In my case, I need to guarantee that the whole asynchronous FSA completes, or I'll risk sending multiple network messages racing and arriving out of order. — kkm inactive - support strike, May 15 '16 at 16:59
@Enigmativity: If you run the sample, remove the `.Window(signal).SelectMany(w => w.Take(1))` part to see that happening. — kkm inactive - support strike, May 15 '16 at 17:07

score 1 · Answer 1 · answered Jun 01 '16 at 20:42

I couldn't find a simple solution since you also want the last value published even if it wasn't in the time frame.

// signal to indicate we want more events to flow
var signal = new BehaviorSubject<bool>(true);
var source = Observable.Interval(TimeSpan.FromMilliseconds(100)).Take(27).Publish();
// Observable to the source but have it skip the last value (would have published the same value twice if not doing this. Ex: if take was 33 instead of 27)
var sequence = source.SkipLast(1).Publish();
// Observable that is just the lastvalue in the sequence
var last = source.PublishLast();

var d = signal.DistinctUntilChanged()
    .Where(on => on) // only let go if we have signal set to true
    .SelectMany(last.Do(_ => signal.OnCompleted()).Merge(sequence).Take(1))  // Side effect of last is to turn off signal
    .Subscribe(
        async ps =>
        {
            signal.OnNext(false); // no more values from the source
            await LengthyWork(ps);
            signal.OnNext(true); // resubscribe to the source
        });

// wire it all  up
last.Connect();
sequence.Connect();
source.Connect();
Thread.Sleep(5000);
Console.WriteLine("Stopping");
d.Dispose();

Part of the idea came from How to turn Off/Restart an Interval? from the MSDN forums.

Thanks, I think I grok the idea. My current working solution involves pretty much low-level implementation, involving locking and requiring a full multithreaded reasoning and therefore complex testing; I do not like it, in short. I am going to see how far your solution could take me. — kkm inactive - support strike, Jun 03 '16 at 06:40

score 0 · Answer 2 · edited May 23 '17 at 11:52

0

You could use the MostRecent operator - I think what you need is a simple version of he answer to this question: Reactive Extensions (Rx) - sample with last known value when no value is present in interval

edited May 23 '17 at 11:52

Community

1
1

answered May 15 '16 at 09:06

Slugart

4,535
24
32

I tried to, but that seems as good as my Window solution at best, since it's a push-to-pull converter, and may repeat the element. It's friend Latest blocks the thread to avoid this, which is not the best way to go (both return an IEnumerable). Also, I cannot think of a way to fit the requirements to observe the last before end element in either solution. – kkm inactive - support strike May 15 '16 at 17:04

Drop any except last sample in observable sequence until consumer is ready

2 Answers2