How to test the performance of Clips?

Question

I have a problem with the performance testing of Clips.What performance needs to be tested about Clips?How do I test its performance?And how do I design some typical rules and data to test the performance of Clips.

I have been learning Clips programming for some time.And I can write some rules. I have used a few rules to test the reasoning time of Clips/6.30,But I don't think it's convincing

score 0 · Answer 1 · answered Jun 16 '19 at 18:48

Performance testing is a very broad topic, but let's split it into two broad categories: specific and general.

If you're testing the performance of a specific program, you run it with a set of representative examples of usage and see if the performance is acceptable.

If you're testing general performance, you're trying to create examples of hypothetical usage that range from best case to worst case scenarios for the algorithms for processing rules. For these cases it's useful to have some understanding of the algorithms used (in the case of CLIPS, the rete algorithm), but in general most rule benchmarks test performance with a large number of rules, a large number of facts, or a combination of the two.

One of the improvements made to CLIPS 6.3 is the use of hash tables for storing partial matches. In CLIPS 6.24, you see the following base line performance for a simple rule that looks for triplets of facts having the same value represented by the variable ?x.

         CLIPS (V6.24 06/15/06)
CLIPS> 
(defrule match
   (a ?x)
   (b ?x)
   (c ?x)
   =>)
CLIPS> (timer (loop-for-count (?i 25) (assert (a ?i) (b ?i) (c ?i))))
0.000200986862182617
CLIPS> (reset)
CLIPS> (timer (loop-for-count (?i 10000) (assert (a ?i) (b ?i) (c ?i))))
19.6043310165405
CLIPS>

Performance in this case does not scale linearly as the number of facts increase by a factor of 400.

In CLIPS 6.3, you see linear scaling for this scenario:

         CLIPS (6.31 5/9/19)
CLIPS> 
(defrule match
   (a ?x)
   (b ?x)
   (c ?x)
   =>)
CLIPS> (timer (loop-for-count (?i 25) (assert (a ?i) (b ?i) (c ?i))))
0.000128
CLIPS> (reset)
CLIPS> (timer (loop-for-count (?i 10000) (assert (a ?i) (b ?i) (c ?i))))
0.049131
CLIPS>

You can find some benchmarks programs for CLIPS at https://sourceforge.net/p/clipsrules/code/HEAD/tree/branches/63x/examples/. These include the widely used waltz and manners benchmarks in addition to a sudoku benchmark and some others in the benchmarks directory.

How to test the performance of Clips?

1 Answers1