Generating random instances of binary Constraint Satisfaction Problems

Question

I'm implementing a generator of random CSPs for doing comparative testing of two different arc-consistency algorithms (AC3 and AC2001). The instances are generated with the parameters of number of variables, domains' size 8same for all variables), number of constraints and the number of values pairs rejected by each constraint (the same for all constraints).

My implementation builds the variables (structure with two fields, name and domain (list)), the constraints 8structure with two fields, variables involved and constraint function)). it creates a hash table that has the variables involved in each restriction as keys and as values a list of the pairs rejected by said constraint. When being used, each constraint checks if the given values to the variables are contained in the rejected values list.

This implementation "works", but very few of the instances generated are in need of arc-reduction, thus they are mostly useless for testing purposes. Here is the code:

; This function generates the complete list of variables for the problem

(defun crea-lista-variables (nvars tdom p-v)
  (loop for i from 0 to (1- nvars) collect
        (crea-variable :nombre i
                       :dominio (crea-dominio-nuevo tdom p-v nil))))

; This function creates a variable's domain, without repetitions. It takes it's
; values from a list of possible values for the problem   

(defun crea-dominio-nuevo (tdom p-v dominio)
  (let ((candidato  (nth (random (1- (length p-v))) p-v)))
    (cond ((= tdom 0) dominio)
          ((not (pertenece candidato dominio))
           (crea-dominio-nuevo (- tdom 1) p-v 
                               (append dominio (list candidato))))
          (t (crea-dominio-nuevo tdom p-v dominio)))))

This function creates the restriccions and it's rejected values

(defun crea-lista-restricciones (nrest npares tdom p-p p-v
                                 restricciones rechazados)
  (let* ((variables (nth (random (1- (length p-p))) p-p))
         (rest (crea-restriccion :variables variables
                                 :funcion #'(lambda (x y &optional (z nil)) 
                                              (not (pertenece (list x y)
                                                              z))))))
    (cond ((= nrest 0) restricciones)
          ((null (gethash variables rechazados))
           (crea-rechazados npares tdom variables p-v rechazados)
           (crea-lista-restricciones (1- nrest) npares tdom  p-p
                                     p-v (append restricciones (list rest))
                                     rechazados))
          (t (crea-lista-restricciones nrest npares tdom  p-p
                                       p-v restricciones rechazados)))))

This function creates the rejected values hash table

(defun crea-rechazados (numpares tamdom variables posibles-valores rechazados)
  (let* ((valor1 (nth (random (1- (length posibles-valores)))
                      posibles-valores))
         (valor2 (nth (random (1- (length posibles-valores)))
                      posibles-valores))
         (candidato (list valor1 valor2))
         (lista (gethash variables rechazados)))
    (cond ((= numpares 0) rechazados)
          ((not (pertenece candidato lista)) 
           (setf (gethash variables rechazados)
                 (append lista (list candidato)))
           (crea-rechazados (1- numpares) tamdom variables
                            posibles-valores rechazados))
          (t (crea-rechazados numpares tamdom variables
                              posibles-valores rechazados)))))

And the main function that creates the global parameters for the solvers to use

(defun genera-problema (numvars tamdom numrest numpares)
  (cond ((< numvars 2) 
         (format t "~&Error: Debe haber al menos 2 variables"))
        ((< tamdom 2)
         (format t "~&Error: Los dominios deben tener al menos dos elementos"))
        ((or (< numrest 0) (> numrest (/ (* numvars (- numvars 1)) 2)))
         (format t "~&Error: numero de restricciones incorrecto"))
        ((or (< numpares 1) (> numpares (- (* tamdom tamdom) 1)))
         (format t "~&Error: numero de pares incorrecto"))
        (t (let ((posibles-valores (loop for i from 0
                                          to (1- (+ tamdom tamdom))
                                         collect i))
                 (posibles-pares (loop for i from 0 to (- numvars 2) append
                                       (loop for j from (+ i 1)
                                               to (1- numvars)
                                             collect (list i j)))))
             (defparameter *RECHAZADOS*
                (make-hash-table :test #'equalp))
             (defparameter *VARIABLES-AC3*
                (crea-lista-variables numvars tamdom posibles-valores))
             (defparameter *VARIABLES-AC2001*
                (loop for variable in *VARIABLES-AC3*
                      collect (crea-variable :nombre (psr-var-nombre var)
                                             :dominio (copia-lista
                                                        (psr-var-dominio var)))))
             (defparameter *RESTRICCIONES*
                (crea-lista-restricciones numrest numpares tamdom
                                          posibles-pares posibles-valores
                                          nil *RECHAZADOS*))
             (defparameter *ARCOS-AC3* 0)
             (defparameter *ARCOS-AC2001* 0)))))

the function "pertenece" checks if an element is in a list I hope is understandable, even with the spanish names. If it's not i can translate it fully.

So, my horrible lisp coding skills aside, is there any error I can fix or improvement I can make in order for the instances generated to be of bigger quality?

Answer 1

There are some remarks about basic coding style:

(loop for i from 0 upto (1- n) ...)

is just

(loop for i below n ...)

crea-dominio-nuevo is a recursive function. In each iteration you add an item to the end of the list. This is the worst possible way to use a singly linked list in Lisp. Appending items to the end of the list should be avoided. There is one cheap operation to add an item, and that is to cons it to the front. If you really really need to add to the end of something, there are two ways to do it:

• use a vector and push to the end

• do the iteration such that you cons to the front and at the end reverse the list once

crea-lista-restricciones has the same problem.

That you call length on lists multiple times is another problem.

Answer 2

In general, when generating benchmarks, one should fix all the parameters and try to play with only one. The latter is usually related to some randomization stuff. In your case, It seems that you don't use any randomization. I strongly suggest that you add for instance a parameter presenting the probability of having two variables together in the scope of a constraint, or how often they are together, etc.

On the other hand, you can use existing benchmarks like http://www.cril.univ-artois.fr/~lecoutre/benchmarks.html. There is a bench of problems, some of them are binaryCSPs (for instance the "mark" problem in the "RAND" cathegory).

Answer 3

Basic fact: Each decision about your random-instances will be reflected in your benchmark. If you want to compare your algorithms A and B, someone could generate random-instances where A (or B) is dominating.

Most of the time, someone is interested in hard instances (because constraint-programming mostly is used for hard problems). In this case, i see two possibilities for you:

1. Easy / Practical approach (like Med proposed): use existing benchmarks. There are competitions evaluating all kind of decision-systems like the CSP Solver Competition. Most of them provide all those instances they used and these instances are very valuable. A lot of time have been going into the generation of these.

2. Hard / Theoretical approach: think about a transformation of 3-SAT -> "your problem"/CP and use randomly generated 3-SAT instances within the phase-transition parameter-space (Mezard, Parisi, Zecchina | 2002). These instances will be very hard (with some theoretical backbone)!