StreamInsight - Problem defining a correct window

Question

I started using StreamInsight and I'm using it as a wcf service. I've already tried seeking help in "A Hitchhiker's Guide to Microsoft StreamInsight Queries" and tried the examples as well as the examples in codeplex.

My problem is this:

My event producer feeds the adapter with AlertEvent's:

public sealed class AlertEvent
{
    public DateTime Date { get; set; }
    public long IDGroup { get; set; }
    public bool IsToNormalize { get; set; }
    public bool IsError { get; set; }
}

When a AlertEvent has IsError = false, the flag IsToNormalize is true;

The behaviour that I'm trying to achieve is when I receive a stream with IsError, I want to see if, in the next 'x' minutes, arrives any alertEvent with IsToNormalize. I then send to output the IsError AlarmEvent that started the search.

What I've done is, when I receive a input that correspond to the filter, I extend its lifetime in 'x' minutes and create a TumblingWindow to see if in that period another AlertEvent arrives with the other flag (using a ExtensionMethod to iterate through all payloads in the window).

var stream= from item in input
            where item.IsError
            group item by item.IdGroup into eachGroup
            from window in eachDigital.AlterEventDuration(e => TimeSpan.FromMinutes((double)1.5)).TumblingWindow(TimeSpan.FromSeconds(15), HoppingWindowOutputPolicy.ClipToWindowEnd)
            select new
            {
                Id = eachDigital.Key,
                NormEvents = window.HasNormalizationEvents(),
                Count = window.Count()
            };

Then, to get the AlarmEvent that triggered the TumblingWindow, I've made a join with the original input.

var resultStream = from e1 in stream
                   join e2 in input
                   on e1.Id equals e2.DigitalTag
                   where e1.NormEvents != 0
                   select e2;

This isn't working at all... :/ Any ideas to help solving this issue?

Another doubt that I have is if there'll be create a new window with a new startDate for each input that passes the filter or will it be created only one tumblingWindow.

Thanks.

@Mitch Wheat: Why would you need that kind of incentive to help another person? If you don't want to help then don't. I don't see the problem... — Megacan, May 27 '11 at 15:19

score 2 · Answer 1 · answered May 27 '11 at 17:44

Try this:

// Move all error events
// to the point where the potential timeout would occur.
var timedOut = input
                   .Where(e => e.IsError == true)
                   .ShiftEventTime(e => e.StartTime + TimeSpan.FromMinutes(5));

// Extend all events IsToNormalize by the timeout.
var following = input
                   .Where(e => e. IsToNormalize == true)
                   .AlterEventDuration(e => TimeSpan.FromMinutes(5));

// Mask away the potential timeout events by the extended events.
// - If IsToNormalize did not occur within the timeout, then the shifted error event
// will remain unchanged by the left-anti-semi-join and represent
// the desired output.
var result = from t in timedOut
             where (from c in following
                    where t.IdGroup == c.IdGroup
                    select c).IsEmpty()
             select t; // or some projection on t

StreamInsight - Problem defining a correct window

1 Answers1