3

I want to have the file locations and character positions of where a macroexpansion takes place in order to highlight the macro expansion in a GUI.

For this, I want to be able to refer to the current position of a macro where an expansion takes place from the macro itself.

For example, if I have the following code:

(defun mve ()
  (magic-macro :maybe :args))

I want to be able to expand it to something like

(defun mve ()
  (progn
    (macro-body-stuff)
    "This expansion took place at #P(myfile.lisp) between chars 16 and 40"))

If such functions existed, a minimal example macro could be something along the lines of

(defmacro maybe-macro (&rest r)
  `(progn
     (macro-body-stuff)
     ,(format nil "This expansion took place at ~S between chars ~D and ~D"
             (??:get-macroexpansion-pathname)
             (??:get-macroexpansion-char-start)
             (??:get-macroexpansion-char-end))))

I am also tagging it as reader-macro because I do no know really where should this take place.

ssice
  • 3,564
  • 1
  • 26
  • 44
  • 1
    With Slime, if you compile code that contains a macro which signals an error at compile-time, you get a compiler note and your code is underlined in red. In order to that, there is a lot of code in swank.lisp and slime.el to get implementation-dependant location information from your lisp. You could probably have a look at it (https://github.com/slime/slime). – coredump May 31 '16 at 11:36
  • *"implementation-dependant location information"* -- this is the main point. It's not portable. – acelent Jun 02 '16 at 09:39

1 Answers1

2

You can't do this portably with a normal macro

"This expansion took place at #P(myfile.lisp) between chars 16 and 40"

In general, you won't be able to get that kind of stuff, because it's not available once the form has been read. E.g,. if you have a file with this content:

;; line 0
(some-form arg1)

and a file with this content:

;; line 0
;; line 1
;; line 2
(


some-form



arg1
                )

Conceptually, the compiler is going to be getting the same input. In principle, the reader first reads a form from the file, and then passes it to the compiler. In both cases, the compiler gets the form (some-form arg1). That's what a macro that you write is guaranteed to have access too. An individual implementation might actually make more available to the compiler, but it will be in an implementation-dependent way, and won't necessarily be exposed to you in a portable way.

There are some standard things that the file loader binds when loading a file that can help in providing some of this information, though. For instance, the load function binds special variables with the pathname and truename of the file:

*load-truename* is bound by load to hold the truename of the pathname of the file being loaded.

*load-pathname* is bound by load to hold a pathname that represents filespec merged against the defaults. That is, (pathname (merge-pathnames filespec)).

The implementation dependent extensions that would provide things like line and column numbers, if any, might be accessible in the same way.

But you can sometimes do this with a reader macro

You can't do this with a normal macro portably, since you don't portably have the mechanism to determine where in the file a form was read from. However, a reader macro invokes a function that gets called with the stream that forms are read from, and there are functions for investigating the position within a stream. For instance, here's a file:

(defparameter *begin* nil
  "File position before reading a form prefixed with #@.")

(defparameter *end* nil
  "File position after reading a form prefixed with #@.")

(eval-when (:compile-toplevel :load-toplevel :execute)
  (set-dispatch-macro-character
   #\# #\@
   (lambda (stream char infix-parameter)
     (declare (ignore char infix-parameter))
     (let ((begin (file-position stream))
           (form (read stream t nil t))
           (end (file-position stream)))
       `(let ((*begin* ,begin)
              (*end* ,end))
          ,form)))))

(defun foo ()
  #@(format nil "form began at ~a and ended at ~a."
            *begin* *end*))

Now, after we load it, we can call foo:

CL-USER> (load ".../reader-macro-for-position.lisp")
T
CL-USER> (foo)
"form began at 576 and ended at 650."

That's a bit brittle of course, because the reader macro could be invoked in a way where the the stream isn't one for which file-position makes a whole lot of sense, so you'd want to do some checking on that, and you still need a way to interpret those file positions in terms of line numbers and columns, but this is a good first shot, I think.

Joshua Taylor
  • 84,998
  • 9
  • 154
  • 353
  • *"because it's not available once the form has been read"* -- this may be true **after** the form has been read, as if passing it to `eval` or in a function/lambda to `compile`, but `compile-file` and `load` can have a specialized reader, or access the internals of the implementation's reader, that mark the current file and where each form starts and ends to later use and/or store as debugging information. – acelent Jun 02 '16 at 09:44
  • @acelent yes, as I said, "So the position information isn't necessarily available. That said, there may be some special variables that the implementation provides to help get some of this information." Implementations very week may have this, but it's not a standardized interface. – Joshua Taylor Jun 02 '16 at 10:33
  • *"**The compiler is actually going to be getting the same input.** The reader first reads a form from the file, and then passes it to the compiler. In both cases, the compiler gets the form (some-form arg1). So the position information isn't necessarily available."* -- The way you phrase it, it seems you're stating that the compiler is provided with a form with no start and end position information (emphasis mine), and it seems you're only referring to the standard `*load-truename*` and `*load-pathname*` (an implementation must bind them while `load`ing, according to the specification). – acelent Jun 02 '16 at 12:11
  • @acelent I see your point; I've updated the wording in my answer. – Joshua Taylor Jun 02 '16 at 12:22
  • @acelent I don't see any problem with leaving it; it helps put the edit in context. – Joshua Taylor Jun 02 '16 at 15:30
  • *"because it's not available once the form has been read"*, what about accessing it while it is being read, is it possible to define a reader-macro what could leave this information available to the compilation step? – ssice Jun 03 '16 at 09:53
  • @ssice That's a good question! It might be possible (perhaps still in an implementation dependent way) since a reader macro gets to look at the input stream, and the input stream might provide access to the position in the file, like [file-position](http://clhs.lisp.se/Body/f_file_p.htm). Actually, I could probably put together an example based on that idea in a little while. – Joshua Taylor Jun 03 '16 at 10:12
  • @ssice Have a look at my update to the answer; I've added a reader-macro based approach. – Joshua Taylor Jun 03 '16 at 12:30
  • @JoshuaTaylor quite nice! I understand the approach (although I think I'd rather override the macros for #\( and #\), because I would rather use s-expressions in this DSL I'm making. However, I don't understand your sentence "*because the reader macro could be invoked in a way where the the stream isn't one for which `file-position` makes a whole lot of sense, so you'd want to do some checking on that*". What do you mean by not making sense? – ssice Jun 03 '16 at 13:53
  • @JoshuaTaylor BTW, the original question just asked for the character position, I actually don't need to get lines/columns (although that's rather easy given the file, and the character position) :-) – ssice Jun 03 '16 at 13:53
  • @ssice For macros on some characters, you may want a look at [How to define symbols that will work like ( and ) by symbol macro?](http://stackoverflow.com/questions/19319277/how-to-define-symbols-that-will-work-like-and-by-symbol-macro/19321377#19321377). – Joshua Taylor Jun 03 '16 at 14:00
  • 1
    @ssice about things not making sense, I just meant that I wasn't sure what results you'd get in something like `CL-USER> (eval (read-from-string "#@(list *begin* *end*)")) => (2 22)`, i.e., where you're not reading from a *file*, so **file-position** might not mean quite as much. – Joshua Taylor Jun 03 '16 at 14:00