0

I have one observable that I use GroupBy on to get a number of streams. I actually want a Scan result over each sub-stream. Let's say the observable is over product prices and the scan result is average price per product type.

I have another stream of events pertaining to those 'products' (let's say "show product price" events) and I want to combine it with the previous stream's latest product price. So the Scan output per group needs to be combined with each element of the event stream to get the latest average price for that event's product.

For some reason I cannot get the right syntax and I have been bashing away at this all day. Can someone please help?


Update

I am adding the code below to illustrate the approximate intent.

 public class Node
{
    private List<int> Details = new List<int>();

    public void AddInfo(int x)
    {
        Details.Add(x );
    }

    public Node(int x)
    {
        Details.Add(x);  
    }

    public int Index => Details[0]%10; //just to simplify the grouping and debugging

    public int Latest => Details.Last();
}

public class Message
{
    private static Random _random = new Random();

    public int MessageNodeInfo { get; private set; }

    public Message()
    {
        MessageNodeInfo = _random.Next(); 
    }
}


public class AccumulatingInfoTest
{


    private static Random _random=new Random();

    private IObservable<Message> MessageStream()
    {
        TimeSpan timeSpan = TimeSpan.FromSeconds(0.5);


        var ret= Observable.Generate(0,
            _ => { return true; }, 
            _ => { return 0; }, 
            _ => { return new Message(); },
            _=> timeSpan)
            .Publish()
            .RefCount();



        return ret;

    }


    public class ArbitraryCommonClass
    {
        public int K { get; set; }
        public Message M { get; set; }
        public Node D { get; set; }

        public ArbitraryCommonClass Combine(ArbitraryCommonClass a)
        {
            return new ArbitraryCommonClass()
            {
                K = this.K,
                M = this.M ?? a.M,
                D = this.D ?? a.D
            };
        }
    }

    public void Start()
    {

        var inputStream = MessageStream();

        inputStream.Subscribe(y => Console.WriteLine("Input: K " + y.MessageNodeInfo % 10 + " V " + y.MessageNodeInfo));


        var nodeInfoStream = inputStream
            .Select(nodeInfo => new Node(nodeInfo.MessageNodeInfo))
            .GroupBy(node => node.Index)
            .Select(groupedObservable => new
                        {
                            Key = groupedObservable.Key,
                            Observable = groupedObservable
                                .Scan(

                                    (nodeAcc, node) => { nodeAcc.AddInfo(node.Latest); return nodeAcc; }

                                    )
                                .Select(a => new ArbitraryCommonClass() { K = a.Index, M = (Message)null, D = a })

                        }
                    );

        var groupedMessageStream =
            inputStream
            .GroupBy(
                    m => new Node(m.MessageNodeInfo).Index
                    )
            .Select(a => new
                        {
                            Key =a.Key,
                            Observable = a.Select(b => new ArbitraryCommonClass() { K = a.Key, M = b, D = null })

                        });



        var combinedStreams = nodeInfoStream
            .Merge(groupedMessageStream)
            .GroupBy(s => s.Key)
            .Select(grp => grp
                .Scan(

                    (state, next) => new { Key = state.Key, Observable = Observable.CombineLatest(state.Observable, next.Observable, (x, y) => { return x.Combine(y); }) }
                )



            )
            .Merge()
            .SelectMany(x => x.Observable.Select(a=>a));

        combinedStreams.Where(x=>x.M!=null).Subscribe(x => Console.WriteLine(x.K + " " + x.M.MessageNodeInfo + " " + x.D.Latest));














    }
}
Sentinel
  • 3,582
  • 1
  • 30
  • 44
  • It would be helpful to post a [mcve]. – Shlomo Apr 17 '18 at 14:33
  • @Shlomo I don't have one. I am trying to work out what that should look like. – Sentinel Apr 17 '18 at 15:07
  • @Shlomo I updated the question with some code to illustrate the approximate intent. It's not quite there (the combine latest is yielding pairs of messages instead of just the one) and it seems overly convoluted and hard to read for what it tries to do. I think I will end up either working with a stateful representation or just adapting what you have done. – Sentinel Apr 18 '18 at 08:37
  • I read through the update, and I'm more confused than before. I'm not really following the problem, or your solution. Hard to understand what you're trying to accomplish. Best of luck though. – Shlomo Apr 18 '18 at 15:06
  • @Shlomo Thanks anyway! I would need to talk through what I have done to explain more, which isn't possible here. It's partially a kind of intellectual exercise for me. In summary, though, in case you end up thinking about this, imagine you have a message come in over TCP (or some other network protocol) and you identify it by that protocol's id (IP address + port). That message is raw binary. It gets handed off to N higher level protocols (eg: Ethereum RLPx), which then hands it off to other protocols, etc.All the while, the same node is being enhanced with new protocol-specific ids.. – Sentinel Apr 18 '18 at 15:24
  • @Shlomo ..and the only thing that ties those ids together is the fact that one protocol 'adapter' accepted the message from the previous one. So we have chains or 'threads' of messages linked by the fact that one protocol handed a message successfully off to another. The collection of such events defines the node completely. There is no need for any other concept of 'node'. No node class even. So if a UI initiated message comes in like "send ETH to Ethereum node XYX", the message can get translated back down the protocol stack to an IP just by using the aggregate stream. Well, anyway. Thx! – Sentinel Apr 18 '18 at 15:27

1 Answers1

1

Assuming the following class:

public class Product
{
    public string Type { get; set; } = "Default";
    public decimal Price { get; set; }
}

Here's a use of GroupBy with Scan (shows the average product price grouped by type). The trick is to Select over the grouped observable to get to the individual groupings, do whatever, then (presumably) merge them back together. You could collapse the Select and the Merge into a single SelectMany, but it can be easier to read when separated:

var productSubject = new Subject<Product>();
var printSignal = new Subject<Unit>();

var latestAverages = productSubject.GroupBy(p => p.Type)
    .Select(g => g
        .Scan((0, 0.0m), (state, item) => (state.Item1 + 1, state.Item2 + item.Price)) //hold in state the count and the running total for each group
        .Select(t => (g.Key, t.Item2 / t.Item1)) //divide to get the average
    )
    .Merge()
    .Scan(ImmutableDictionary<string, decimal>.Empty, (state, t) => state.SetItem(t.Key, t.Item2)); //Finally, cache the average by group.


printSignal.WithLatestFrom(latestAverages, (_, d) => d)
    .Subscribe(avgs =>
    {
        foreach (var avg in avgs)
        {
            Console.WriteLine($"ProductType: {avg.Key}. Average: {avg.Value}");
        }
        Console.WriteLine();
    });

var productsList = new List<Product>()
{
    new Product { Price = 1.00m },
    new Product { Price = 2.00m },
    new Product { Price = 3.00m },

    new Product { Price = 2.00m, Type = "Alternate" },
    new Product { Price = 4.00m, Type = "Alternate" },
    new Product { Price = 6.00m, Type = "Alternate" },
};

productsList.ForEach(p => productSubject.OnNext(p));

printSignal.OnNext(Unit.Default);
productSubject.OnNext(new Product { Price = 4.0m });
printSignal.OnNext(Unit.Default);
productSubject.OnNext(new Product { Price = 8.0m, Type = "Alternate" });
printSignal.OnNext(Unit.Default);

This uses nuget package System.Collections.Immutable.

Shlomo
  • 14,102
  • 3
  • 28
  • 43
  • Thanks a lot Shlomo! I am away from PC now but will check in the coming hours. – Sentinel Apr 17 '18 at 16:20
  • I had a quick check (I will play with this more in my morning tomorrow) and if I understand correctly, maintaining the whole dictionary was kind of what I was hoping to avoid. I will try and put together an alternative code chunk in an update to the question, one of the many versions that for some reason I could not get to work. One version, that I could not figure out, was how to say get observable A to "fan out" into groups by key K, observable B to "fan out" into groups by key K (of same type) and then combine the groupedovservables by K. I just cant see it. – Sentinel Apr 17 '18 at 20:36
  • How do you want to combine A-Fan(K) and B-Fan(K)? Let's say you have observable A `IGroupedObservable(Key == x)` and observable B of type `IGroupedObservable(Key == x)`. What do you want to do with them? – Shlomo Apr 17 '18 at 21:06
  • OK so for A-Fan I (could, depending on implementation) want a scan to yield an anonymous type with two properties, key K and aggregated value over T1. For B-Fan just each new item in each groupedobservable to yield new anon type of T2 value, K, and latest aggregate T1 from A-fan of index K. So essentially an arbitrary combine of fan b withlatestfrom equivalent fan a. Will try to post code tomorrow. Problem I had was that query would not invoke the subscribe on fan a when I combined, so nothing fed through. Could be a case of brain fade on my part, just could not get it to work. – Sentinel Apr 17 '18 at 21:26
  • For what it is worth, I am trying to replace a stateful representation of p2p network nodes (a dictionary of nodes) with a stream representation of nodes, where nodes are described as an aggregate over node related information discovered and emitted from an underlying stream of messages exchanged over different underlying protocols.Node network identifier eg IPv4 is K and the aggregate function is other protocol ids, eg Kademlia id, depending on what emerges in protocol related message exchangs – Sentinel Apr 17 '18 at 21:35