is there an upper limit for the length function on lists in Frege?

Question

For the real-world-frege project I did the exercise from real-world-haskell where the assignment is to create a length function for lists and to compare it against the internal length function.

My proposed solution is under https://github.com/Dierk/Real_World_Frege/blob/master/realworld/chapter3/I_Exercise_1.fr

The beef of it is:

mylength :: [a] -> Int

mylength (_:xs)       = 1 + (mylength xs)
mylength []           = 0

-- optLength uses tail recursion and forces eager argument evaluation
optLength xs = privateLength xs 0 where
    privateLength (_:rest) !count = privateLength rest (count + 1)
    privateLength []       !count = count

main _ = do
    assert (mylength [])  (length [])  "empty"
    assert (mylength [0]) (length [0]) "one element"
--  assert (mylength [0..30_000]) (length [0..30_000]) "many elements lead to stack overflow"
    assert (optLength [0..1_000_000])  (length [0..1_000_000])  "optLength can do a little more"
    assert (optLength [0..10_000_000]) (length [0..10_000_000]) "this takes 40 seconds (20 s each)"
--  println (length [0..100_000_000]) -- never stops

Both my and the internal length function work fine for lists under 1 million entries, get very slow with 10 M and appear to not stop at all for 100 M.

Frege's internal length function (not Haskell's) appears to have an upper limit below 100 million. Is that so?

Please insert the relevant code inline so the question can be understood without reading other websites - it's fine to keep the link for background, but the question should stand alone. — Ganesh Sittampalam, Jun 26 '14 at 17:57
What is your actual question? Are you asking why yours is slow for large lists? — bheklilr, Jun 26 '14 at 18:08
Your `myLength` builds a chain of unevaluated thunks, while the `optLength` forces the evaluation of these thunks. Try using a strict `foldl`. — cdk, Jun 26 '14 at 18:10
What? `length [0..10^8]` evaluates properly: http://ideone.com/waTfCp — effectfully, Jun 26 '14 at 18:55

score 1 · Accepted Answer · answered Jun 28 '14 at 13:09

Two issues here.

mylength suffers of not being tail recursive and thus should stack overflow.

Here is proof that there is no limit per se for the length function (should also hold for your optlength function):

enter image description here

(The TimeoutException is due to limits from the cloud service, not from Frege.)

But, you are passing length [0..100_000_000] to assert and it looks like assert takes their arguments also lazily. Unfortunately, this causes the reference to the start of the list to get kept alive. This, in turn, causes the whole list to get realized in memory, and thus things get very slow, probably because your heap is on the limit and the GC tries to find some free space for the next argument.

This is the same problem as in ths issue: https://github.com/Frege/frege/issues/65

Go give it a try with -Xmx4g, which I am sure is no issue on your shiny premium laptop :)

The general solution would be to force the result before calling assert like:

let 
    n = length [0..100_000_000]
    m = optlength [0..100_000_000] 
in n `seq` m `seq` assert n m "should be equal"

Superb answer - as always :-) Thanks a lot. – Dierk Jun 29 '14 at 13:55 — Dierk, Jun 29 '14 at 13:55

is there an upper limit for the length function on lists in Frege?

1 Answers1