Common Lisp nested structural changes

Question

I'm learning Common Lisp at the moment, and trying to send some JSON to a web service. The JSON that I am going to send is represented in a structure similar to the following:

((:ITEMS
  ((:KEY . "value1") (:IGNORE 1 2 3))
  ((:KEY . "value2") (:IGNORE 1 2 3))))

I'd like to remove the key :IGNORE from every item in :ITEMS.

In Clojure (with which I'm more familiar) I'd use update to apply a functional change to the key :ITEMS, map over each item and then use select-keys or dissoc to get the structure that I need. I can't quite find the right way to perform this modification in Common Lisp however.

Should I use copy-tree to duplicate the structure and then use setf to change the tree?

NB1: I'm not sure if the data structure that I'm using is 'good', but it's what's returned from my call to json:decode-json-from-string from the CL-Json package.

So far I have this

(SETF (CDR (ASSOC :ITEMS JSON))
    (REMOVE-IF (LAMBDA (E) (EQUAL (CAR E) :IGNORE)) (CDR (ASSOC :ITEMS JSON))))

which is wrong (E is an alist with the form ((:KEY . "value1") (:IGNORE 1 2 3))) but I can't see how to go about making my code right.

NB2: This is how I'd do it in Clojure:

(update json :items #(map (fn [x] (dissoc x :ignore)) %))

jkiiski · Answer 1 · 2017-09-13T13:10:28.137

The other answers already show ways to do this with alists, but alternatively you could customize the json decoder to return a more convenient datastructure. Since you want to use them functionally/persistently, the FSet library might be a good choice.

(ql:quickload '(cl-json :fset :alexandria))
(use-package :alexandria)

First, let's customize the decoder to turn json objects into FSET:MAPs, and json arrays to FSET:SEQs.

(defvar *current-array* nil)
(defun beginning-of-array ()
  (setf *current-array* (fset:empty-seq)))
(defun array-member (item)
  (fset:push-last *current-array* item))
(defun end-of-array ()
  *current-array*)

(defvar *current-object* nil)
(defvar *current-object-key* nil)
(defun beginning-of-object ()
  (setf *current-object* (fset:empty-map)))
(defun object-key (key)
  (setf *current-object-key*
        (funcall json:*identifier-name-to-key*
                 (funcall json:*json-identifier-name-to-lisp* key))))
(defun object-value (value)
  (fset:includef *current-object* *current-object-key* value))
(defun end-of-object ()
  *current-object*)

(defun call-with-fset-decoder-semantics (function)
  (let ((json:*aggregate-scope-variables*
          (list* '*current-array* '*current-object* '*current-object-key*
                 json:*aggregate-scope-variables*)))
    (json:bind-custom-vars (:beginning-of-array #'beginning-of-array
                            :array-member #'array-member
                            :end-of-array #'end-of-array
                            :beginning-of-object #'beginning-of-object
                            :object-key #'object-key
                            :object-value #'object-value
                            :end-of-object #'end-of-object)
      (funcall function))))

(defmacro with-fset-decoder-semantics (&body body)
  `(call-with-fset-decoder-semantics (lambda () ,@body)))

Assuming your JSON looks something like this.

(defvar *json* "{
                  \"something\": \"foo\",
                  \"items\": [
                    {
                      \"key\": \"value1\", 
                      \"ignore\": [1, 2, 3]
                    },
                    {
                      \"key\": \"value2\", 
                      \"ignore\": [1, 2, 3]
                    }
                  ],
                  \"other\": \"bar\"
                }")

Now you can decode it into FSet structures. Notice that FSet uses its custom reader syntax for printing objects. #{| (key value)* |} means a FSET:MAP and #[ ... ] is a FSET:SEQ. You can use FSET:FSET-SETUP-READTABLE if you want to be able to read them, but it's not necessary. You should create a custom readtable before doing that.

(with-fset-decoder-semantics
  (json:decode-json-from-string *json*))
;=> #{|
;      (:ITEMS
;       #[
;         #{| (:KEY "value1") (:IGNORE #[ 1 2 3 ]) |}
;         #{| (:KEY "value2") (:IGNORE #[ 1 2 3 ]) |} ])
;      (:OTHER "bar")
;      (:SOMETHING "foo") |}

You can use FSET:IMAGE to remove the :IGNORE key from the items with FSET:LESS. FSET:WITH adds a key to the map, or replaces the existing one.

(let ((data (with-fset-decoder-semantics
              (json:decode-json-from-string *json*))))
  (fset:with data :items (fset:image (rcurry #'fset:less :ignore)
                                     (fset:lookup data :items))))
;=> #{|
;      (:ITEMS #[ #{| (:KEY "value1") |} #{| (:KEY "value2") |} ])
;      (:OTHER "bar")
;      (:SOMETHING "foo") |}

FSet also handles modify macros conveniently:

(let ((data (with-fset-decoder-semantics
              (json:decode-json-from-string *json*))))
  (fset:imagef (fset:lookup data :items) (rcurry #'fset:less :ignore))
  data)
;=> #{|
;      (:ITEMS #[ #{| (:KEY "value1") |} #{| (:KEY "value2") |} ])
;      (:OTHER "bar")
;      (:SOMETHING "foo") |}

This might look at first glance as if it was modifying the data destructively, but it actually isn't.

(let* ((data (with-fset-decoder-semantics
               (json:decode-json-from-string *json*)))
       (old-data data))
  (fset:imagef (fset:lookup data :items) (rcurry #'fset:less :ignore))
  (values data old-data))
;=> #{|
;      (:ITEMS #[ #{| (:KEY "value1") |} #{| (:KEY "value2") |} ])
;      (:OTHER "bar")
;      (:SOMETHING "foo") |}
;=> #{|
;      (:ITEMS
;       #[
;         #{| (:KEY "value1") (:IGNORE #[ 1 2 3 ]) |}
;         #{| (:KEY "value2") (:IGNORE #[ 1 2 3 ]) |} ])
;      (:OTHER "bar")
;      (:SOMETHING "foo") |}

Svante · Answer 2 · 2017-09-12T23:27:10.777

As a quick shot, here is an incomplete attempt of a port of Clojure's update:

(defun update (a-list keyword fun)
  "Creates a new a-list from the given a-list where the item under the given
keyword is replaced by the value created by applying fun to it.  Expects the
a-list to have only keywords as keys and those keywords to be unique."
  (loop :for el :in a-list
        :for (a . d) := el
        :if (eq a keyword)
          :collect (cons a (funcall fun d))
        :else
          :collect el))

In your case, it could be used like this:

(update '((:ITEMS ((:KEY . "value1") (:IGNORE 1 2 3))
                  ((:KEY . "value2") (:IGNORE 1 2 3))))
        :items
        (lambda (items)
          (mapcar (lambda (item)
                    (remove :ignore item :key #'car))
                  items)))

returning:

((:ITEMS ((:KEY . "value1")) ((:KEY . "value2"))))

(This should be improved a bit, e. g. shortcutting as soon as the key was found, allowing for other keys, parameterizing the usual key and test functions etc.)

coredump · Accepted Answer · 2017-09-13T20:08:29.840

The basic non-destructive way to do this is:

(let ((data '((:ITEMS
               ((:KEY . "value1") (:IGNORE 1 2 3))
               ((:KEY . "value2") (:IGNORE 1 2 3))))))
  (acons :items
         (mapcar (lambda (alist)
                   (remove :ignore alist :key #'car))
                 (cdr (assoc :items data)))
         (remove :items data :key #'car)))

=> ((:ITEMS ((:KEY . "value1")) ((:KEY . "value2"))))

The last call to "remove" is not strictly necessary; if you leave "data" instead, you will keep a copy of the previous :items in the resulting list. Whether you want to do that or not depends on your needs.

Clojurify all the things

(defun arem (key &optional (alist nil alistp))
  (if alistp
      #1=(remove key alist :key #'car)
      (lambda (alist) #1#)))

(defun update-fn (fn key &optional (alist nil alistp))
  (if alistp
      #1=(acons key (funcall fn (cdr (assoc key alist))) (arem key alist))
      (lambda (alist) #1#)))

Now it looks like this:

(update-fn (lambda (u) (mapcar (arem :ignore) u))
           :items
           '((:ITEMS
              ((:KEY . "value1") (:IGNORE 1 2 3))
              ((:KEY . "value2") (:IGNORE 1 2 3)))))

Finally:

(defmacro each (expr &aux (list (gensym)))
  `(lambda (,list)
     (mapcar ,expr ,list)))

Which gives:

(update-fn (each (arem :ignore))
           :items
           '((:ITEMS
              ((:KEY . "value1") (:IGNORE 1 2 3))
              ((:KEY . "value2") (:IGNORE 1 2 3)))))

=> ((:ITEMS ((:KEY . "value1")) ((:KEY . "value2"))))

You can also write:

(update-fn (each (arem :ignore)) :items)

... and you obtain a closure.

Common Lisp nested structural changes

3 Answers3

Clojurify all the things