3

The SMTLib2 directive (get-info all-statistics) displays several numbers, e.g.

num. conflicts:     4
num. propagations:  0 (binary: 0)
num. qa. inst:      23

In order to test different axiomatisations and encodings I'd like to know which of those numbers are appropriate to declare that one Z3 run is better/more efficient than another.

Guessing from the names I'd say that num. qa. inst - the number of quantifier instantiations - is a good indicator (lower = better), but what about the others?

Leonardo de Moura
  • 21,065
  • 2
  • 47
  • 53
Malte Schwerhoff
  • 12,684
  • 4
  • 41
  • 71

1 Answers1

6

Number of quantifier instantiations is a good measure for checking whether your axiomatisation is producing too many instances or not. You can also use QI_PROFILE=true. It will produce statistics for each quantifier. You can use the attribute :qid to give a name to a quantifier. You may also use DEFAULT_QID=true, and Z3 will produce a name based on line numbers. QI_PROFILE_FREQ= will display the statistics after every instances are generated. These options are useful for detecting instantiations loops.

"num. conflicts" is useful for estimating the size of the search space traversed by Z3. We may say an axiomatisation is "better" if the size of the search space is smaller.

Cheers, Leo

Leonardo de Moura
  • 21,065
  • 2
  • 47
  • 53
  • 1
    Thanks for pointing out the additional options! But what about the other numbers, e.g. "num. decisions"? Which of them should be considered when benchmarking Z3 encodings? – Malte Schwerhoff Jul 28 '11 at 07:26
  • 2
    "num. conflicts" is much more relevant than "num. decisions". BTW, Z3 uses non-chronological backtracking. Thus, n decisions does not imply that Z3 explored 2^n branches. "num. conflicts" is roughly equal to the number of branches explored by Z3. There are other numbers that are useful for detecting bottlenecks in theory reasoning. For example, "pivots" --> Z3 is having trouble with linear arithmetic, "gomory cuts" --> integer arithmetic, "interface eqs" --> theory combination, etc. – Leonardo de Moura Aug 01 '11 at 17:26