Why does it cons so much?

Question

When I run the following codes in SBCL (2.3.5), I am surprised by how much bytes consed.

(defun number-of-digits (num)
  (do ((n 1 (1+ n))
       (num (floor num 10) (floor num 10)))
      ((zerop num) n)))

(defun p25 ()
  (do ((a 0 (+ a b))
       (b 1 a)
       (n 0 (1+ n)))
      ((>= (number-of-digits a) 1000) n)))

(time (p25))
Evaluation took:
  0.890 seconds of real time
  0.888141 seconds of total run time (0.878463 user, 0.009678 system)
  [ Run times consist of 0.004 seconds GC time, and 0.885 seconds non-GC time. ]
  99.78% CPU
  1,864,981,440 processor cycles
  368,895,776 bytes consed
  
4782

Obviously, the codes itself does not "cons" any list. So my questions are:

1. Why there are so much bytes consed? Where does it come from?

2. How to eliminate the "consed bytes"? I mean it is just numeric calculation, I think there might be ways to make the code not to cons any byte (or just a few bytes).

Answer 1

Let's look the largest number generated:

CL-USER 44 > (integer-length 1070066266382758936764980584457396885083683896632151665013235203375314520604694040621889147582489792657804694888177591957484336466672569959512996030461262748092482186144069433051234774442750273781753087579391666192149259186759553966422837148943113074699503439547001985432609723067290192870526447243726117715821825548491120525013201478612965931381792235559657452039506137551467837543229119602129934048260706175397706847068202895486902666185435124521900369480641357447470911707619766945691070098024393439617474103736912503231365532164773697023167755051595173518460579954919410967778373229665796581646513903488154256310184224190259846088000110186255550245493937113651657039447629584714548523425950428582425306083544435428212611008992863795048006894330309773217834864543113205765659868456288616808718693835297350643986297640660000723562917905207051164077614812491885830945940566688339109350944456576357666151619317753792891661581327159616877487983821820492520348473874384736771934512787029218636250627816)
3319

CL-USER 45 > (ceiling * 8)
415

So Lisp will need roughly 415 8bit bytes to represent this number on the heap.

Your code creates a lot of these numbers and each number is a new object on the heap.

Alternatively, for example, you could write your own array-based numerics, where you use arrays of digits to represent numbers. These arrays could be reused in the computation. You would need a + operation and you would need to get the number of digits in the array.

Answer 2

Common Lisp uses fixnums for integers in some implementation-defined range, and bignums for integers that are outside of the fixnum range. Using bignums involves consing by the implementation.

The do loop in p25 iterates until the number of digits in a is one thousand or more. This is well outside of the range of a fixnum (which is about 64 bits in SBCL). You can inspect most-positive-fixnum to see the largest representable fixnum on your system. Once a passes this threshold it is treated as a bignum, and consing begins. Of course, b will also be subject to this bignum treatment.

Working with very large numbers requires handling their representations, and if you don't want to use bignums you will have to implement your own bignum-like facility.