Poor Drools performance when retracting facts

Question

We are doing scalability testing on Drools 7.5 with a single rule defined:

...
declare RuleEvent
    @role(event)
    @timestamp(timestamp)
    entityId : String
    metrics : Map
    timestamp : long
end

declare window AlarmWindow
    RuleEvent() over window:time( 5m )
    from entry-point "vmStream"
end

rule "rule0"
when
    $rec : RuleEvent( $entityId : entityId) from window AlarmWindow
then
    Alarm alarm = new Alarm();
    alarmOutQueue.put(alarm);
end

The KieBase is configured for EventProcessingOption.STREAM and the SessionPseudoClock. We have -Xmx12g Java option set and there doesn't appear to be any memory pressure.

100,000 RuleEvents are inserted at a time representing batches of values taken at 1 minute intervals.

session.fireAllRules() takes around 35 seconds for the first 5 iterations (representing 5 minutes of simulated data), but subsequent iterations take 8 1/2 MINUTES. Presumably drools is doing the retraction of the expired 100,000 facts automatically due to expiration of the sliding window. We can observe similar results with explicitly removing the 100,000 expired facts using session.delete(fact) before calling fireAllRules.

So Drools seems able to handle insertion of facts fine, but retraction is really bogging down. Does anyone have suggestions?

Yes I did. When facts outside the time window need to be retracted, the SlidingTimeWindow class gets notified and removes the fact from the queue. But the SlidingTimeWindow class is using a PriorityQueue that is keyed on the timestamp and not the fact i.d. This results in a sequential scan of the PriorityQueue and then a rearrangement of the binary tree FOR EACH event retracted. This is fairly easy to fix, I haven't gotten around to submitting my patch. In one particular test case, the overall time was 720 seconds before to 7 seconds afterwards putting Drools on par with alternative CEPs. — user2405341, Mar 20 '18 at 17:20

Poor Drools performance when retracting facts

0 Answers0