StreamInsight query based on holding period

Question

I have an event stream which contains the values for some variables. The values change every second. Each variable has high limit, low limit and a threshold period defined.

For example, say Variable1 has high limit = 95, low limit = 5 and threshold = 5 seconds.

So if the values for Variable1 stay above 95 (high limit) for the threshold period (5 seconds), then we should create an output event.

I am able to create query for checking the individual events (if they have crossed the limits), but I am not sure how to create a windowed query to check if the limits were violated for the threshold period.

Thanks,

Answer 1

Well, not entirely a Linq query, but should do the trick:

public IEnumerable<IEnumerable<int>> GetWindows(IEnumerable<int> events)
{
    while (events.Any())
    {
        events = events.SkipWhile(x => x > 5 && x < 95);
        if (events.Any())
        {
            var isLow = events.First() <= 5;
            var res = events.TakeWhile(x => isLow ? x <= 5 : x >= 95).ToList();
            if (res.Count >= 5)
                yield return res;
            events = events.Skip(res.Count);
        }
    }
}

Basically:

• Skip all uninteresting events
• Take the window and yield it if more entries than threshold period
• Advance the enumeration by skipping the window

The you can use it like this:

foreach (var window in GetWindows(events))
{
   ... raise event
}

Note: If your events come in as a potentially infinite stream and your event windows can get very large then you won't get the window returned until it's fully consumed which might take some time and memory. You could break the TakeWhile part after having seen 5 elements and then yield there and simply skip the rest of the window afterwards. It depends a bit of what you want to do exactly.