4

I have a database of business events our web application produce.
I want to write JavaScript application that will take those events (several hundreds at most) and using Complex Event Processing will find some patterns in them.

For example, if we had login-failure event without login event in 15 minutes we want to know about it. We correlate between events using session ID.

I prefer to use exist library, from what I saw RxJS seem to be the right tool.
My question is how to build the stream of events using the timestamp of the original event and not the current computer time?

I saw that RxJS has operators of time but it looks like it is using the current time.

Update 1
I've found HistoricalScheduler class which looks like a good directions but there is no documentation about it and I'm not sure it is even exist in RxJS.

Ido Ran
  • 10,584
  • 17
  • 80
  • 143
  • Because these events have already happened, and you most likely intend to run this query as a one-off kind of deal, I doubt Rx (or any kind of observable) is the way you want to go. It would make more since to use Ix (interactive extensions), because the events your want to query already exist. – cwharris Nov 24 '13 at 23:12
  • Furthermore, what environment are you running in? There may be much better ways to do this kind of query. For instance, you might be better off using a simple db query. Is there a reason you need to pull all of the results into memory, as using RxJS (without a query provider) would do? – cwharris Nov 24 '13 at 23:13
  • Lastly, for reasons previously mentioned, I believe Schedulers have little to nothing to do with this logic. Schedulers control when and where events/notifications *will* be processed, and are not intended to indicate when data *was* processed. In fact, the timestamp of each event should have been stored in your database. – cwharris Nov 24 '13 at 23:17
  • I understand and agree. I thought it should be easy to use Rx to query over exist events, apparently I was wrong. When watching and reading about Rx I read they use virtual schedulers for testing, it seem like a good way to re-run exist events as if they really happening. Last thing is if this system exist turn it into real-time processor should be as easy as change the input for it, does it? – Ido Ran Nov 25 '13 at 15:05
  • Ahh, I see what you mean. Actually, if you wanted to do it that way, I'd suggest using the actual timestamp of the event (not the system time, or scheduler time) for your logic. Because that timestamp already exists (if it doesn't you can't use Rx to get accurate timestamps of past events), you can just use it directly... you won't need schedulers or anything like that. Just sort the events by time before yielding them. – cwharris Nov 25 '13 at 20:18

2 Answers2

1

Presumably, you should be able to sort the existing events by the timestamp for when they occurred, and merge that with the future events (since they will obviously be sorted already). After which you can work with the events in the same manner.

var pastEvents = [
    { userId: 1, time: 100, status: 'failure' },
    { userId: 2, time: 400, status: 'success' },
    { userId: 1, time: 300, status: 'success' },
    { userId: 4, time: 200, status: 'success' },
  ]
  .sort(function (a, b) { return b.time - a.time; });

var futureEvents = getFutureEventObservable();

var events = Rx.Observable.fromArray(pastEvents).merge(futureEvents);

var groups = events.groupBy(function (x) { return x.userId; });

// do "complex event processing" here.

Unfortunately, if the events do not already have timestamps, that data is lost, and Rx has no special way of reproducing it.

cwharris
  • 17,835
  • 4
  • 44
  • 64
1

Check the rxmarbles example code.

Especifically: https://github.com/staltz/rxmarbles/blob/master/src/controllers/utils.coffee which uses the Rx.VirtualTimeScheduler.

From my understanding you should be able to adapt that to fit your use case

Thomas Haferlach
  • 86
  • 4