Is there an Awk- or Lisp-like programming language that can process a stream of s-expressions?

Question

I have been creating some PCB footprints in KiCad recently, which are stored in s-expression files with data that looks like this:

(fp_text user %R (at 0 5.08) (layer F.Fab)
  (effects (font (size 1 1) (thickness 0.15)))
)
(fp_line (start -27.04996 -3.986) (end -27.24996 -3.786) (layer F.Fab) (width 0.1))
(pad "" np_thru_hole circle (at 35.56 0) (size 3.175 3.175) (drill 3.175) (layers *.Cu *.Mask)
  (clearance 1.5875))
(pad 96 smd rect (at 1.25 3.08473) (size 0.29972 1.45034) (layers F.Cu F.Paste F.Mask)
  (clearance 0.09906))

I would like to be able to write shell one-liners to efficiently edit multiple parameters. I would normally use Awk for something like this, but the recursive nature of s-expressions makes it ill-suited for the task. I would like to know if there is a programming language with an interpreter designed to handle piped data and can process s-expressions natively. Perhaps a data-driven dialect of Lisp would do this, but I'm not sure where to look.

In summary, I would like to be able to make quick edits to an s-expression file in a similar manner to the way Awk lets me process columns of data line-by-line; only in the case of s-expressions the processing would be performed level-by-level.

Example: find all of the pad expressions of type smd with (size 0.29972 1.45034), and renumber each one based its position.

Answer 1

Simple script

Here is an example in Common Lisp, assuming your input is in file "/tmp/ex.cad" (it could also be obtained by reading the output stream of a process).

The main processing loop consists in opening the file in order to obtain an input stream in (which is automatically closed at the end of with-open-file), loop over all forms in the file, process them and possibly output them to standard output. You could complexify the process as much as you want, but the following is good enough:

(with-open-file (in #"/tmp/ex.cad")
  (let ((*read-eval* nil))
     (ignore-errors
       (loop (process-form (read in))))))

Suppose you want to increase the width of fp_line entries, ignore fp_text and otherwise print the form unmodified, you could define process-form as follows:

(defun process-form (form)
  (destructuring-bind (header . args) form
    (print
     (case header
       (fp_line (let ((width (assoc 'width args)))
                  (when width (incf (second width) 3)))
                form)
       (fp_text (return-from process-form))
       (t form)))))

Running the previous loop would then output:

(FP_LINE (START -27.04996 -3.986) (END -27.24996 -3.786) (LAYER F.FAB) (WIDTH 3.1)) 
(PAD "" NP_THRU_HOLE CIRCLE (AT 35.56 0) (SIZE 3.175 3.175) (DRILL 3.175) (LAYERS *.CU *.MASK) (CLEARANCE 1.5875)) 
(PAD 96 SMD RECT (AT 1.25 3.08473) (SIZE 0.29972 1.45034) (LAYERS F.CU F.PASTE F.MASK) (CLEARANCE 0.09906)) 

More safety

From there, you can build more elaborate pipelines, with the help of pattern matching or macros if you want. You have to take into account some safety measures, like binding *read-eval* to nil, using with-standard-io-syntax and binding *print-circte* to T as suggested by tfb, disallowing fully qualified symbols (by having #\: signal an error), etc. Ultimately, like Shell scripts one-liners, the amount of precautions you add is based on how much you trust your inputs:

;; Load libraries
(ql:quickload '(:alexandria :optima))

;; Import symbols in current package
(use-package :optima)
(use-package :alexandria)

;; Transform source into a stream
(defgeneric ensure-stream (source)
  (:method ((source pathname)) (open source))
  (:method ((source string)) (make-string-input-stream source))
  (:method ((source stream)) source))

;; make reader stop on illegal characters    
(defun abort-reader (&rest values)
  (error "Aborting reader: ~s" values))

Dedicated package for KiCad symbols (exporting is optional):

(defpackage :kicad
  (:use)
  (:export #:fp_text
           #:fp_line
           #:pad
           #:size))

Loop over forms:

(defmacro do-forms ((form source &optional result) &body body)
  "Loop over forms from source, eventually return result"
  (with-gensyms (in form%)
    `(with-open-stream (,in (ensure-stream ,source))
       (with-standard-io-syntax
         (let ((*read-eval* nil)
               (*print-circle* t)
               (*package* (find-package :kicad))
               (*readtable* (copy-readtable)))
           (set-macro-character #\: #'abort-reader nil)
           (loop
              :for ,form% := (read ,in nil ,in)
              :until (eq ,form% ,in)
              :do (let ((,form ,form%)) ,@body)
              :finally (return ,result)))))))

Example:

;; Print lines at which there is a size parameter, and its value
(let ((line 0))
  (labels ((size (alist) (second (assoc 'kicad:size alist)))
           (emit (size) (when size (print `(:line ,line :size ,size))))
           (process (options) (emit (size options))))
    (do-forms (form #P"/tmp/ex.cad")
      (match form
        ((list* 'kicad:fp_text _ _ options) (process options))
        ((list* 'kicad:fp_line options) (process options))
        ((list* 'kicad:pad _ _ _ options) (process options)))
      (incf line))))

Output

(:LINE 2 :SIZE 3.175) 
(:LINE 3 :SIZE 0.29972)

Answer 2

Just write a simple Lisp or Scheme script which loops on reading and processes recursively your s-expr as required. On Linux I would recommend using Guile (a good Scheme interpreter) or perhaps Clisp (a simple Common Lisp implementation) or even SBCL (a very powerful Common Lisp).

(You might consider DSSSL, but in your case it is overkill)

Notice that your sample input is not an S-expression, because (layer F.Fab) is not one (since after the dot you should have another s-expression, not an atom like Fab). I guess it is a typo and should be (layer "F.Fab"); or maybe your KiCad software don't process S-expressions, but some other input language (which should be specified, probably in EBNF notation) inspired by S-expressions.

Notice also that KiCad is a free software and has a community with forums and a mailing list. Perhaps you should ask your actual problem there?

PS. We don't know what transformation you have in mind, but Scheme and Common Lisp are really fit for such tasks. In most cases they are extremely simple to code (probably a few lines only).