0

I am writing a program that opens a lisp file, calls "read" on the stream until the stream is empty, and does things with the lists it collects.

This was working quite nicely until I discovered that "read" will perform package lookup, for instance if it encounters some-package:foo it will complain that Package SOME-PACKAGE does not exist.

Here is an example showing what I mean:

(read (make-string-input-stream "(list 'foo :foo some-package:foo)"))

So I now I would like one of three things:

  1. Make it so "read" will ignore package namespaces so I can convert arbitrary source files to lists of symbols.
  2. Use some other parsing library with similar behavior to "read" but that only gets plain symbols, either by mangling the : or ignoring the colon and everything before it.
  3. Pre-processing the file and use regex or such to package lookups and replace them with plain names, such as converting "some-package:foo" to simply "foo"

The purpose of all of this in the first place was to make a function call dependency graph. I'm aware there exists things of that nature of much higher quality that exist, but I wanted to do it myself for fun/learning. However, I have hit a snag with this problem and don't know how to proceed.

Robert Williams
  • 1
  • case 3 will be buggy - because the package name helps to distinguish name collisions (in the case your code uses a function of the same name of another package) so better to change `"some-package:foo"` to `"some-package-foo"` isn't it? However then a bug will appear if there is a part of a program after a `(in-package some-package)` ... – Gwang-Jin Kim Nov 03 '18 at 06:16
  • the strategy a which `read` is following, is explained here https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node188.html – Gwang-Jin Kim Nov 03 '18 at 06:21
  • if you read http://norvig.com/lispy.html you see how to make a lisp reader in python ... – Gwang-Jin Kim Nov 03 '18 at 07:04
  • how about if it parses everything to string tokens? – Gwang-Jin Kim Nov 03 '18 at 08:39

2 Answers2

1

For your use case, you could handle the package-error condition by creating the required package and restarting. That would also preserve the symbol identities. Note that you need to handle in-package forms when you encounter them.

Svante
  • 50,694
  • 11
  • 78
  • 122
1

The simplest answer is to tell the Lisp reader to read colon #\: as is:

(defun read-standalone-char (stream char)
  (declare (ignore stream))
  char)

(defun make-no-package-prefix-readtable (&optional (rt (copy-readtable)))
  "Return a readtable for reading while ignoring package prefixes."
  (set-syntax-from-char #\: #\Space rt)
  (set-macro-character #\: #'read-standalone-char nil rt)
  rt)

(let ((*readtable* (make-no-package-prefix-readtable)))
  (read-from-string "(list 'foo :foo some-package:foo)"))
==> (LIST 'FOO #\: FOO SOME-PACKAGE #\: FOO) ; 33

The obvious problem is that this will read FOO:BAR and FOO :BAR identically, but you might be able to work around that.

sds
  • 58,617
  • 29
  • 161
  • 278
  • He could make ':' a constituent character with (set-syntax-from-char #\A #\:) and then find the ':' character within the string if further processing is needed. – Leo Nov 05 '18 at 08:12
  • @Leo: first, you have the wrong arg order in your `set-syntax-from-char` call. Second, it does not work, alas. – sds Nov 05 '18 at 13:41