Is it possible to sample the latest item in a hot observable grouped by key?

Question

In Rx.NET, is it possible to sample the latest item in a hot observable grouped by key?

For example, if I have an IObservable<Price>, where Price is:

Price 
- Key
- Bid
- Offer

Let's assume that the IObservable is linked to an external price feed.

Am I able to retrieve all latest Prices, grouped by Key, sampled every 1 second using Rx?

Answer 1

Assuming some observable source, this returns all prices grouped and sampled by key that have come in in the last second.

var sampled = source
    .GroupBy(p => p.Key)
    .SelectMany(o => o.Sample(TimeSpan.FromSeconds(1)));

If there's some price which hasn't received a message in the last second, that will not be included.

If you want old prices included, this will work:

var sampled2 = source
    .Scan(ImmutableDictionary<int, Price>.Empty, (state, p) => state.SetItem(p.Key, p))
    .Replay(1)
    .RefCount();
var dummySubscription = sampled2.Subscribe();
var result = Observable.Interval(TimeSpan.FromSeconds(1))
    .SelectMany(_ => sampled2.Take(1).SelectMany(state => state.Values));

Just make sure to dispose of the DummySubscription when done with the result observable.

Answer 2

Does this do what you want?

IObservable<ImmutableDictionary<string, Price>> sampled =
    Observable
        .Create<ImmutableDictionary<string, Price>>(o =>
        {
            var output = ImmutableDictionary<string, Price>.Empty;
            return
                source
                    .Do(x => output = output.SetItem(x.Key, x))
                    .Select(x => Observable.Interval(TimeSpan.FromSeconds(1.0)).Select(y => output).StartWith(output))
                    .Switch()
                    .Subscribe(o);
        });

Answer 3

A hot observable won't keep any old values in memory but you could catch the last price of each known key yourself, for example in a dictionary.

Please refer to the following sample code.

Dictionary<string, double> _prices = new Dictionary<string, double>();

GetPrices()
    .Buffer(TimeSpan.FromSeconds(1))
    .Subscribe(prices =>
    {
        if (prices != null && prices.Count > 0)
        {
            var grouped = prices.GroupBy(x => x.Key);
            foreach (var group in grouped)
                _prices[group.Key] = group.Last().Bid;
        }

        //print out the last quote of each known price key
        foreach (var price in _prices)
        {
            Console.WriteLine("Key: " + price.Key + ", last price: " + price.Value);
        }
    });

It should print the last quote of each known key every second.

Answer 4

Here is a polished version of Shlomo's idea, of maintaining the latest values per key using the Scan operator and an ImmutableDictionary as state. The custom operator below (SampleByKey) samples a sequence of key-bearing elements at a specific interval. Upon each sampling tick an IDictionary<TKey, TSource> is emitted, containing the latest values that have been emitted so far by each key.

public static IObservable<IDictionary<TKey, TSource>> SampleByKey<TSource, TKey>(
    this IObservable<TSource> source,
    Func<TSource, TKey> keySelector,
    TimeSpan interval,
    IEqualityComparer<TKey> keyComparer = default)
{
    return source
        .Scan(ImmutableDictionary.Create<TKey, TSource>(keyComparer),
            (dict, x) => dict.SetItem(keySelector(x), x))
        .Publish(published => Observable
            .Interval(interval)
            .WithLatestFrom(published, (_, dict) => dict)
            .TakeUntil(published.LastOrDefaultAsync()));
}

Usage example:

IObservable<IDictionary<string, Price>> sampled = priceFeed
    .SampleByKey(p => p.Key, TimeSpan.FromSeconds(1.0));

In case that zero elements have been emitted by the source during two samplings, the same dictionary will be emitted consecutively.

This implementation is very similar with one I've posted previously in a question, about how to sample a sequence at a dynamically changeable time interval.

Note: I scraped my previous implementation (Revision 1) as too complex and potentially leaky. Shlomo's approach is easier to understand and modify as needed.