Common Lisp and ASDF System Testing

Question

I am wondering how one runs tests using ASDF. Suppose we have the following system definition. What command would I run to run the tests?

I have tried running things like (asdf:compile-system cl-mongo) and (asdf:load-system cl-mongo) but it does not become apparent how to test. If it helps, the code comes from this repository.

Should I somehow be compiling and loading the test system as well?

   (in-package #:cl-user)

(defpackage #:cl-mongo-system (:use #:cl #:asdf))

(in-package #:cl-mongo-system)

(asdf:defsystem #:cl-mongo
  :name   "cl-mongo"
  :author "Fons Haffmans; fons.haffmans@gmail.com"
  :version "0.7"
  :licence "MIT"
  :description "lisp system to interact with mongodb, a non-sql db"
  :depends-on (:uuid
           :babel
           :bordeaux-threads
           :documentation-template
           :lisp-unit
           :parenscript
           :split-sequence
           :usocket)
  :serial t
  :components 
 ((:module "src"
    :serial t
    :components ((:file "packages")
         (:file "octets")
         (:file "pair")
         (:file "encode-float")
         (:file "bson-oid")
         (:file "bson-binary")
         (:file "bson-time")
         (:file "bson-regex")
         (:file "bson-code")
         (:file "bson")
         (:file "bson-decode")
         (:file "bson-array")
         (:file "document")
         (:file "mongo-syntax")
         (:file "java-script")
         (:file "bson-encode-container")
         (:file "protocol")
         (:file "mongo")
         (:file "db")
         (:file "mem")
         (:file "do-query")
         (:file "doc")
         (:file "map-reduce")
         (:file "shell")))
   (:static-file "README.md")
   (:static-file "COPYING")))

(asdf:defsystem #:cl-mongo-test
  :name   "cl-mongo"
  :author "Fons Haffmans; fons.haffmans@gmail.com"
  :version "0.7"
  :licence "MIT"
  :description "testing cl-mongo"
  :depends-on (:cl-mongo)
  :serial t
  :components 
  ((:module "test"
    :serial t
    :components ((:file "package")
         (:file "test-utils")
         (:file "regression")))))

As requested, I am including the error I receive when I run (asdf: compile-system 'cl-mongo-test)

COMPILE-FILE-ERROR while
compiling #<CL-SOURCE-FILE "cl-mongo-test" "test" "regression">
   [Condition of type UIOP/LISP-BUILD:COMPILE-FILE-ERROR]

Restarts:
 0: [RETRY] Retry compiling #<CL-SOURCE-FILE "cl-mongo-test" "test" "regression">.
 1: [ACCEPT] Continue, treating compiling #<CL-SOURCE-FILE "cl-mongo-test" "test" "regression"> as having been successful.
 2: [RETRY] Retry ASDF operation.
 3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
 4: [RETRY] Retry SLIME REPL evaluation request.
 5: [*ABORT] Return to SLIME's top level.
 --more--

Backtrace:
  0: (UIOP/LISP-BUILD:CHECK-LISP-COMPILE-RESULTS NIL T T "~/asdf-action::format-action/" ((#<ASDF/LISP-ACTION:COMPILE-OP > . #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-mongo-test" "test" "regression">)))
  1: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unavailable argument> #<unavailable argument> #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-mongo-test" "test" "regression">)
  2: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-mongo-test" "test" "regression">) [fast-method]
  3: ((:METHOD ASDF/PLAN:PERFORM-PLAN (LIST)) ((#1=#<ASDF/LISP-ACTION:COMPILE-OP > . #2=#<ASDF/LISP-ACTION:CL-SOURCE-FILE #3="cl-mongo" #4="src" "shell">) (#5=#<ASDF/LISP-ACTION:LOAD-OP > . #2#) (#1# . #6=..
  4: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
  5: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) ((#1=#<ASDF/LISP-ACTION:COMPILE-OP > . #2=#<ASDF/LISP-ACTION:CL-SOURCE-FILE #3="cl-mongo" #4="src" "shell">) (#5=#<ASDF/LISP-ACTION:LOAD-OP > . #2#) (#1# ..
  6: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
  7: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {10049ED323}>) [fast-method]
  8: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/SYSTEM:SYSTEM "cl-mongo-test">) [fast-method]
  9: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/SYSTEM:SYSTEM "cl-mongo-test">)
 10: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
 11: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/SYSTEM:SYSTEM "cl-mongo-test">) [fast-method]
 12: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:COMPILE-OP CL-MONGO-TEST)
 13: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
 14: (ASDF/CACHE:CALL-WITH-ASDF-CACHE #<CLOSURE (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10049CF6AB}> :OVERRIDE NIL :KEY NIL)
 15: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:COMPILE-OP CL-MONGO-TEST) [fast-method]
 16: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:COMPILE-OP CL-MONGO-TEST) [fast-method]
 17: (ASDF/OPERATE:COMPILE-SYSTEM CL-MONGO-TEST)
 18: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ASDF/OPERATE:COMPILE-SYSTEM (QUOTE CL-MONGO-TEST)) #<NULL-LEXENV>)
 19: (EVAL (ASDF/OPERATE:COMPILE-SYSTEM (QUOTE CL-MONGO-TEST)))
 --more--

did you read README file from repository? it states *"The cl-mongo-test package contains regression tests for all the features currently supported. cl-mongo-test is asdf-installable, but does not export it's tests. To run the quick test, do this : (use-package :cl-mongo-test) (quick-test)"*. is it not working for you? — rsm, Feb 11 '18 at 01:41
Yes. When I follow these instructions, I get an ASDF error, which are not very descriptive. Hence, I was wondering if I was doing something incorrectly or there was a problem with the system itself. — MadPhysicist, Feb 11 '18 at 01:47
can you please edit your question and tell us something more about errors you get? — rsm, Feb 11 '18 at 01:49
Are you interested in that particular system or how testing is supposed to be done in ASDF in general? Usually you would just do `(asdf:test-system system-name)`, but the system you gave as an example does not define a test-op. See [Alexandria](https://gitlab.common-lisp.net/alexandria/alexandria) for a better example. — jkiiski, Feb 11 '18 at 09:34
Did you get some more descriptive compilation errors printed out that aren’t included in the error which was signalled? — Dan Robertson, Feb 11 '18 at 12:23
It should be noted that this ASD does not follow current best practices for ASDF. In fairness, those best practices have changed/sharpened in recent years. Please take a look at the ASDF manual which describes them quite clearly. — Svante, Feb 11 '18 at 19:52
@jkiiski I am interested in both. When I write my systems, I set up testing as describe in Ehvince's answer below, but I was not sure how the system in question was set up. — MadPhysicist, Feb 11 '18 at 22:27

Ehvince · Answer 1 · 2018-02-21T16:52:19.457

2

When I create a new project with cl-project the asdf definition contains a :in-order-to ((test-op …)) clause:

(asdf:defsystem :foo
   :author ""
    …
   :in-order-to ((test-op (test-op :foo.test))))

and my foo.test.asdf:

(asdf:defsystem :foo.test
  :author ""
   …
  :perform (test-op (op system)
             (funcall (read-from-string "prove:run")
                      (system-relative-pathname :foo.test "test/"))))

To run tests, we can execute asdf:test-system or (in this case, using prove prove:run in your REPL.

https://lispcookbook.github.io/cl-cookbook/testing.html

edited Feb 21 '18 at 16:52

answered Feb 11 '18 at 14:32

Ehvince

17,274
7
58
79

1

Fore beginners, just in case it wasn't clear: asdf isnt tied to a specific test framework (like prove/parachute), in the above answer we use asdf's test-op feature to call `prove:run`, but it could be used to call any code whether another testing framework like [5am](https://common-lisp.net/project/fiveam/) or even your own code :) – Baggers Feb 21 '18 at 09:40
1

Also: the reason you need to use `read-from-string`, or, more commonly, `symbol-call`, is that the test library is not yet loaded when the ASD file is executed, neither the system under test. – Renato Jul 01 '23 at 17:11

score 1 · Answer 2 · answered Feb 21 '18 at 19:51

1

See the asdf best practices document: https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md

answered Feb 21 '18 at 19:51

Faré

952
5
4

Common Lisp and ASDF System Testing

2 Answers2