Adding events to the beginning of an Rx observable using ReplaySubject - possible?

Question

I am implementing a bus using Rx (reactive framework) and so far it looks good. The problem that I am facing right now is how to add events before the beginning of a stream.

To give more context, this is for an Event Source (a la CQRS/ES). When the bus starts, a number of subscribers get the IObservable. At this point, the bus asks the subscribers what event number/sequence they need to start at. The events are loaded from disk starting at the lowest number and added to the stream, with each subscriber using a where statement to start at the right event. As the application runs, new events get added and go to the subscribers. This part works great.

Some subscribers that attach late, but still need all the events. It looks like ReplaySubject fits the bill, as long as the number of events is small enough. I am thinking that I can probably do some disk/offline caching to keep more them around (any pointers welcome!).

The bigger issue is that when some subscribers get the IObservable, they need to get events that occurred before the ones that were loaded initially. A situation like the following.

When the bus started it loaded event #50 onward (the earliest anyone wanted). Now, a new subscriber requests the IObservable, except they need from #20 onward. So what I want to do is load 20-49, and add them before the start of the stream. None of the existing subscribers should see any of the events (it will be filtered by the Where).

It seems like this should be possible, but I can't quite get my head around it. Is this possible in Rx? If so, how?

Thanks! Erick

score 3 · Accepted Answer · answered May 12 '12 at 07:19

3

I would look at this in a very simple Rx way. Don't use a ReplaySubject, use a single Subject for all of the "live" events and concat the historical events in front of this for each subscriber.

I would imagine that you might have some sort of data structure that keeps track of the historical records. If you've loaded/captured events from #50 to say #80 in memory already, then when a new subscriber comes along that requests the events from #20 I'd do something like this:

var fromIndex = 20;
var eventSubject = new Subject<Event>();
capturedEvents
    .GetFromIndex(fromIndex) // Gets a enumerable list of events
    .ToObservable()
    .Concat(eventSubject)
    .Subscribe( ... );

You then can have the eventSubject be used to capture new events, like so:

eventSubject.Subscribe(capturedEvents.CaptureEvent);

Does this meet your need?

answered May 12 '12 at 07:19

Enigmativity

113,464
11
89
172

I think it does - I need to test it in the morning. I have to say that working with Rx takes a little mental adjustment! – Erick T May 12 '12 at 07:40
1

@ErickT - I agree is does take some mental adjustment, but if you're like me you'll find that you do not want to go back to the previous way of doing things. I just hate, for example, using regular event handlers now. So much code is made short, succinct, and safe using Rx. – Enigmativity May 12 '12 at 09:18
I have something that is basically what you have above, but I am not seeing the old events arrive. Do I have to have to subscriptions? – Erick T May 19 '12 at 00:00
@ErickT - It should work fine - you get no values unless you have subscriptions. Perhaps email me the code to james at enigmativity dot com? – Enigmativity May 19 '12 at 02:04

score 2 · Answer 2 · answered May 12 '12 at 06:00

2

How about just:

itemsFromTwentyToFourtyNine.Concat(currentItems);

answered May 12 '12 at 06:00

Ana Betts

73,868
16
141
209

Adding events to the beginning of an Rx observable using ReplaySubject - possible?

2 Answers2