Passing compile-time state between nested macros in Clojure

Question

I'm trying to write a macro that can be used both in a global and nested way, like so:

;;; global:
(do-stuff 1)

;;; nested, within a "with-context" block:
(with-context {:foo :bar}
  (do-stuff 2)
  (do-stuff 3))

When used in the nested way, do-stuff should have access to {:foo :bar} set by with-context.

I've been able to implement it like this:

(def ^:dynamic *ctx* nil)

(defmacro with-context [ctx & body]
  `(binding [*ctx* ~ctx]
     (do ~@body)))

(defmacro do-stuff [v]
  `(if *ctx*
     (println "within context" *ctx* ":" ~v)
     (println "no context:" ~v)))

However, I've been trying to shift the if within do-stuff from runtime to compile-time, because whether do-stuff is being called from within the body of with-context or globally is an information that's already available at compile-time.

Unfortunately, I've not been able to find a solution, because nested macros seem to get expanded in multiple "macro expansion runs", so the dynamic binding of *ctx* (as set within with-context) is not available anymore when do-stuff gets expanded. So this does not work:

(def ^:dynamic *ctx* nil)

(defmacro with-context [ctx & body]
  (binding [*ctx* ctx]
    `(do ~@body)))

(defmacro do-stuff [v]
  (if *ctx*
    `(println "within context" ~*ctx* ":" ~v)
    `(println "no context:" ~v)))

Any ideas how to accomplish this?

Or is my approach totally insane and there's a pattern for how to pass state in such a way from one macro to a nested one?

EDIT:

The body of with-context should be able to work with arbitrary expressions, not only with do-stuff (or other context aware functions/macros). So something like this should also be possible:

(with-context {:foo :bar}
  (do-stuff 2)
  (some-arbitrary-function)
  (do-stuff 3))

(I'm aware that some-arbitrary-function is about side effects, it might write something to a database for example.)

Answer 1

When the code is being macroexpanded, Clojure computes a fixpoint:

(defn macroexpand
  "Repeatedly calls macroexpand-1 on form until it no longer
  represents a macro form, then returns it.  Note neither
  macroexpand-1 nor macroexpand expand macros in subforms."
  {:added "1.0"
   :static true}
  [form]
    (let [ex (macroexpand-1 form)]
      (if (identical? ex form)
        form
        (macroexpand ex))))

Any binding you establish during the execution of a macro is no more in place when you exit your macro (this happens inside macroexpand-1). By the time an inner macro is being expanded, the context is long gone.

But, you can call macroexpand directly, in which case the binding are still effective. Note however that in your case, you probably need to call macroexpand-all. This answer explains the differences between macroexpand and clojure.walk/macroexpand-all: basically, you need to make sure all inner forms are macroexanded. The source code for macroexpand-all shows how it is implemented.

So, you can implement your macro as follows:

(defmacro with-context [ctx form]
  (binding [*ctx* ctx]
    (clojure.walk/macroexpand-all form)))

In that case, the dynamic bindings should be visible from inside the inner macros.

Answer 2

I'd keep it simple. This is solution avoids state in an additional *ctx* variable. I think it is a more functional approach.

(defmacro do-stuff 
  ([arg1 context]
    `(do (prn :arg1 ~arg1 :context ~context))
         {:a 4 :b 5})
  ([arg1]
    `(prn :arg1 ~arg1 :no-context)))

(->> {:a 3 :b 4}
     (do-stuff 1)
     (do-stuff 2))

output:

:arg1 1 :context {:a 3, :b 4}
:arg1 2 :context {:b 5, :a 4}

Answer 3

there is one more variant to do this, using some macro magic:

(defmacro with-context [ctx & body]
  (let [ctx (eval ctx)]
    `(let [~'&ctx ~ctx]
       (binding [*ctx* ~ctx]
         (do ~@body)))))

in this definition we introduce another let binding for ctx. Clojure's macro system would then put it into the &env variable, accessible by the inner macros at compile-time. Notice that we also keep bindings so that inner functions could use it.

now we need to define the function to get the context value from macro's &env:

(defn env-ctx [env]
  (some-> env ('&ctx) .init .eval))

and then you can easily define do-stuff:

(defmacro do-stuff [v]
  (if-let [ctx (env-ctx &env)]
    `(println "within context" ~ctx ":" ~v)
    `(println "no context:" ~v)))

in repl:

user> (defn my-fun []
        (println "context in fn is: " *ctx*))
#'user/my-fun

user> (defmacro my-macro []
        `(do-stuff 100))
#'user/my-macro

user> (with-context {:a 10 :b 20}
        (do-stuff 1)
        (my-fun)
        (my-macro)
        (do-stuff 2))
;;within context {:a 10, :b 20} : 1
;;context in fn is:  {:a 10, :b 20}
;;within context {:a 10, :b 20} : 100
;;within context {:a 10, :b 20} : 2
nil

user> (do (do-stuff 1)
          (my-fun)
          (my-macro)
          (do-stuff 2))
;;no context: 1
;;context in fn is:  nil
;;no context: 100
;;no context: 2
nil