Prolog Out of stack error

Question

I'm working on Problem 26 from 99 Prolog Problems:

P26 (**) Generate the combinations of K distinct objects chosen from the N elements of a list

Example:
?- combination(3,[a,b,c,d,e,f],L).

L = [a,b,c] ;

L = [a,b,d] ;

L = [a,b,e] ;

So my program is:

:- use_module(library(clpfd)).

combination(0, _, []).
combination(Tot, List, [H|T]) :- 
    length(List, Length), Tot in 1..Length,
    append(Prefix, [H], Stem), 
    append(Stem, Suffix, List), 
    append(Prefix, Suffix, SubList),
    SubTot #= Tot-1,
    combination(SubTot, SubList, T).

My query result starts fine but then returns a Global out of stack error:

?- combination(3,[a,b,c,d,e,f],L).
L = [a, b, c] ;
L = [a, b, d] ;
L = [a, b, e] ;
L = [a, b, f] ;
Out of global stack

I can't understand why it works at first, but then hangs until it gives Out of global stack error. Happens on both SWISH and swi-prolog in the terminal.

Did you try doing a `trace`? You'll probably find that, once your program finds solutions, it keeps making new, ever growing lists with `append` that it can try in order to find additional solutions which it won't ever satisfy. Your first `append(Prefix, [H], Stem)` has two variables, so those will keep growing unbounded. — lurker, Mar 11 '17 at 03:58
@lurker, yes, I've seemed to have `trace`d, `debug`ged, `guitrace`d everything and the program always hangs after generating `L = [a, b, f] ;` until the Out of stack error (actually with `debug` as a first clause on the query then the error just never comes and it hangs forever). If I put `trace` before `append(Prefix, [H], Stem)` I get the following output: `Call:lists:append(_14580, [_14486], _14584)` `Call:lists:append(_14592, [_14498], _14596)` `Call:lists:append(_14604, [_14510], _14608)` `L = [a, b, c]` `L = [a, b, d]` `L = [a, b, e]` `L = [a, b, f]` `Out of global stack` — , Mar 11 '17 at 04:06

CapelliC · Accepted Answer · 2017-03-11T08:05:33.630

2

if you try to input, at the console prompt, this line of your code, and ask for backtracking:

?- append(Prefix, [H], Stem).
Prefix = [],
Stem = [H] ;
Prefix = [_6442],
Stem = [_6442, H] ;
Prefix = [_6442, _6454],
Stem = [_6442, _6454, H] ;
...

maybe you have a clue about the (main) problem. All 3 vars are free, then Prolog keeps on generating longer and longer lists on backtracking. As Boris already suggested, you should keep your program far simpler... for instance

combination(0, _, []).
combination(Tot, List, [H|T]) :- 
    Tot #> 0,
    select(H, List, SubList),
    SubTot #= Tot-1,
    combination(SubTot, SubList, T).

that yields

?- aggregate(count,L^combination(3,[a,b,c,d,e],L),N).
N = 60.

IMHO, library(clpfd) isn't going to make your life simpler while you're moving your first steps into Prolog. Modelling and debugging plain Prolog is already difficult with the basic constructs available, and CLP(FD) is an advanced feature...

edited Mar 11 '17 at 08:05

answered Mar 11 '17 at 07:59

CapelliC

59,646
5
47
90

Thank you. First I selected this answer for the informative query `?- append(Prefix, [H], Stem)` that helped illustrate the issue. Second, I was unaware of the `select` predicate, that definitely helps. Third, why is CLP(FD) an advanced feature? The SWI-Prolog documentation actually says the opposite, that CPL(FD) is a good default and lower-level predicates should be left to advanced users: http://www.swi-prolog.org/pldoc/man?section=clpfd – Mar 11 '17 at 19:38
Prolog it's worth to study itself. CLP(FD) is based on different concepts, and is not available in many Prolog implementations. I think that a good understanding of Prolog is a prerequisite to use CLP(FD), since several basic features do **not** mix well. Solving problems with Prolog will not teach how to solve in CLP(FD), and viceversa. Indeed, a lot of other CP (Constraint Programming) implementations exist, hosted by (usually) procedural languages. It's a big deal, really... – CapelliC Mar 11 '17 at 20:56
1

@RobertL.SimioneII The whole chapter on CLP(FD) in the SWI-Prolog implementation is written by the author of library(clpfd). I am not convinced that there is a universal agreement whether CLP(FD) is a good default for integer arithmetic; it definitely imposes a run time penalty for cases where `is/2` and friends would suffice. My personal and biased opinion is that CLP(FD) is great if you understand its limitations and trade-offs and know _when_ and _why_ to use it. – Mar 12 '17 at 08:33
@RobertL.SimioneII Let's just say that like _any_ useful abstraction, CLP(FD) is a _leaky_ abstraction. For reasons that go into philosophical and ideological territory, some computer scientists think it is OK to only teach the leaky abstraction. – Mar 12 '17 at 08:40
@Boris: thanks for introducing me to the [leaky abstraction](https://en.wikipedia.org/wiki/Leaky_abstraction) concept. It summarizes well my feelings about this topic (I'm always envy of your control of English language :). – CapelliC Mar 12 '17 at 08:53

score 2 · Answer 2 · answered Mar 11 '17 at 23:15

I can't understand why it works at first, but then hangs until it gives Out of global stack error.

The answers Prolog produces for a specific query are shown incrementally. That is, the actual answers are produced lazily on demand. First, there were some answers you expected, then a loop was encountered. To be sure that a query terminates completely you have to go through all of them, hitting SPACE/or ; all the time. But there is a simpler way:

Simply add false at the end of your query. Now, all the answers are suppressed:

?- combination(3,[a,b,c,d,e,f],L), false.
ERROR: Out of global stack

By adding further false goals into your program, you can localize the actual culprit. See below all my attempts: I started with the first attempt, and then added further false until I found a terminating fragment (failure-slice).

combination(0, _, []) :- false.                % 1st
combination(Tot, List, [H|T]) :-
    length(List, Length), Tot in 1..Length,    % 4th terminating
    append(Prefix, [H], Stem), false,          % 3rd loops
    append(Stem, Suffix, List), false,         % 2nd loops
    append(Prefix, Suffix, SubList),
    SubTot #= Tot-1, false,                    % 1st loops
    combination(SubTot, SubList, T).

To remove the problem with non-termination you have to modify something in the remaining visible part. Evidently, both Prefix and Stem occur here for the first time.

score 0 · Answer 3 · edited May 23 '17 at 12:26

The use of library(clpfd) in this case is very suspicious. After length(List, Length), Length is definitely bound to a non-negative integer, so why the constraint? And your Tot in 1..Length is weird, too, since you keep on making a new constrained variable in every step of the recursion, and you try to unify it with 0. I am not sure I understand your logic overall :-(

If I understand what the exercise is asking for, I would suggest the following somewhat simpler approach. First, make sure your K is not larger than the total number of elements. Then, just pick one element at a time until you have enough. It could go something like this:

k_comb(K, L, C) :-
    length(L, N),
    length(C, K),
    K =< N,
    k_comb_1(C, L).

k_comb_1([], _).
k_comb_1([X|Xs], L) :-
    select(X, L, L0),
    k_comb_1(Xs, L0).

The important message here is that it is the list itself that defines the recursion, and you really don't need a counter, let alone one with constraints on it.

select/3 is a textbook predicate, I guess you should find it in standard libraries too; anyway, see here for an implementation.

This does the following:

?- k_comb(2, [a,b,c], C).
C = [a, b] ;
C = [a, c] ;
C = [b, a] ;
C = [b, c] ;
C = [c, a] ;
C = [c, b] ;
false.

And with your example:

?- k_comb(3, [a,b,c,d,e,f], C).
C = [a, b, c] ;
C = [a, b, d] ;
C = [a, b, e] ;
C = [a, b, f] ;
C = [a, c, b] ;
C = [a, c, d] ;
C = [a, c, e] ;
C = [a, c, f] ;
C = [a, d, b] ;
C = [a, d, c] ;
C = [a, d, e] ;
C = [a, d, f] ;
C = [a, e, b] ;
C = [a, e, c] . % and so on

Note that this does not check that the elements of the list in the second argument are indeed unique; it just takes elements from distinct positions.

This solution still has problems with termination but I don't know if this is relevant for you.

Thank you Boris. `Tot in 1..Length` is a check that you aren't asking for more items from the list than there are items in the list, as you pointed out "First, make sure your K is not larger than the total number of elements". Also, I was unaware of the select predicate, that definitely helps. I think your point that "The important message here is that it is the list itself that defines the recursion, and you really don't need a counter, let alone one with constraints on it." was both the important and tricky part of the exercise for me. — , Mar 11 '17 at 19:55
@RobertL.SimioneII There is `between(1, Length, Tot)` if you know that `Length` is an integer. — , Mar 12 '17 at 08:25

Prolog Out of stack error

3 Answers3