What is the scalability of Clips..that is what is the maximum limit on the number of rules and facts in CLIPS ? I am using CLIPS 6.24 but unable to upload one million facts!!
Asked
Active
Viewed 883 times
1
-
Actually I was planning on using CLIPS to extract information from a large data set.For example :- if we have facts in the form of (name college place) then (?name collegeA ?place) would return all the people belonging to collegeA. Will this be more feasible instead of using traditional RDBMS?(Given that CLIPS may or may not handle large datasets) – – code_Assasin May 25 '16 at 10:52
-
Matching the conditions of a single rule to a collection of facts is similar to making a database query. When you have multiple rules, a rule engine such as CLIPS adds two benefits. First, since you've got multiple conditions you're making multiple queries so the rule engine will share similar portions of the queries to provide better performance. Second, you're making the same set of queries repeatedly as each rule is allowed to execute, so the rule engine remembers the query results and incrementally reevaluates the portions of the queries that are affected. – Gary Riley May 25 '16 at 16:48
-
@GaryRiley thanks a lot for your help.Very happy to receive help from expert of expert systems.I still have few lingering questions : 1. Will CLIPS be a better option than RDBMS or a graphical Database for this kind of data? and 2. I also came across a few articles on neural expert systems .I didnt a very clear idea on them ,do you have any resources on that? – code_Assasin May 26 '16 at 11:06
-
If all you're doing is making queries on a large predefined data set, I'd use a RDBMS. – Gary Riley May 26 '16 at 16:51
1 Answers
1
The implementation of the rete algorithm in CLIPS shares common patterns between rules, so pattern matching performance will scale reasonably as you add rules. The worst case performance will occur when no patterns are shared between rules.
A large number of improvements were also made in version 6.3 to improve performance for large numbers of facts:
CLIPS (V6.24 06/15/06)
CLIPS> (timer (loop-for-count (?i 100000) (assert (data ?i))))
3.78414106369019
CLIPS> (reset)
CLIPS> (timer (loop-for-count (?i 1000000) (assert (data ?i))))
885.884355068207
CLIPS>
CLIPS (6.30 3/17/15)
CLIPS> (timer (loop-for-count (?i 100000) (assert (data ?i))))
0.136654
CLIPS> (reset)
CLIPS> (timer (loop-for-count (?i 1000000) (assert (data ?i))))
6.046085
CLIPS>
I haven't spent much time optimizing the loading of large rule sets, so it's certainly possible that this or other related development functionality doesn't scale well. I worked on an expert system using a commercial product that had a few thousand simple rules. Performance and compilation time were acceptable and probably would have scaled well, but synchronization between the desktop client and rule repository was painfully slow and buggy. I can't imagine trying to scale that application without improvements to that component of the product.
There's a large number of factors that affect scalability beyond the number of fact and rules, notably the design of the rules, their complexity, and the relationship between the facts and the rule patterns. Your best bet when evaluating a tool is to write a simple program that generates a large number of rules similar to the ones you envision writing and associated facts that will match those rules and use those to test scalability.
The largest CLIPS system that I've written had around 600 complex rules, processed tens of thousands of facts, and had acceptable performance with CLIPS 6.24. I don't recall anyone mentioning significantly larger systems, but practically I don't think you can scale to a million facts with 6.24 unless you're generous with what's acceptable performance.
With 6.3, I think it's plausible that you can build a system with at least several thousand rules and a few hundred thousand facts, but again your best bet is to programmatically generate rules and facts and test the performance.
Gary Riley
- 10,130
- 2
- 19
- 34