Parsing an infixed string with a macro

Question

I am trying to evaluate an infixed expression in a string.

Some sample data to evaluate my code against:

(def data {:Location "US-NY-Location1"
           :Priority 3})

(def qual "(Location = \"US\")")

I would like the qual string to be converted to something like this form and evaluated by clojure:

(= (:Location data) "US")

I wrote the following macro to achieve this:

(defmacro parse-qual [[data-key op val] data-map]
  `(~op ((keyword (str (quote ~data-key))) ~data-map) ~val))

and a helper function:

(defn eval-qual [qual-str data]
  (eval `(parse-qual ~(clojure.edn/read-string qual-str) ~data)))

(eval-qual qual data) provides me with the expected result

This is the first macro I have written and I am still trying to wrap my head around all the quoting and unquoting.

1. I want to know if there is a more efficient way to achieve the above? (Or even without the need for a macro at all)

2. How can I expand the macro to deal with nested expressions. To handle an expression like ((Location = "US") or (Priority > 2)). Any pointers would be appreciated. I am currently trying to play with tree-seq to solve this.

3. How can I make this more robust and be more graceful in case of an invalid qual string.

I also wrote a second iteration of the parse-qual macro as follows:

(defmacro parse-qual-2 [qual-str data-map]
  (let [[data-key op val] (clojure.edn/read-string qual-str)]
    `(~op ((keyword (str (quote ~data-key))) ~data-map) ~val)))

and on macroexpand throws the following:

playfield.core> (macroexpand `(parse-qual-2 qual data))
java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to java.lang.String

And I am at a loss on how to debug this!

Some extra information:

macroexpand of parse-qual on the REPL gives me the following:

playfield.core> (macroexpand
 `(parse-qual ~(clojure.edn/read-string qual) data))

(= ((clojure.core/keyword (clojure.core/str (quote Location))) playfield.core/data) "US")

Thank you @Alan Thompson, I was able to write this as a functions as follows, this also allows for nested expressions to be evaluated.

(def qual "(Location = \"US\")")
(def qual2 "((Location = \"US\") or (Priority > 2))")
(def qual3 "(Priority > 2)")
(def qual4 "(((Location = \"US\") or (Priority > 2)) and (Active = true))")

(defn eval-qual-2 [qual-str data]
  (let [[l op r] (clojure.edn/read-string qual-str)]
    (cond
      (and (seq? l)
           (seq? r)) (eval (list op (list eval-qual-2 (str l) data) (list eval-qual-2 (str r) data)))
      (seq? l)       (eval (list op (list eval-qual-2 (str l) data) r))
      (seq? r)       (eval (list op (list (keyword  l) data) (list eval-qual-2 (str r) data)))
      :else          (eval (list op (list (keyword  l) data) r)))))

(eval-qual-2 qual data) ; => false
(eval-qual-2 qual2 data) ; => true
(eval-qual-2 qual3 data) ; => true
(eval-qual-2 qual3 data) ; => true

Answer 1

You don't need or want a macro for this. A plain function can process data like this.

Macros are only for transforming source code - you are effectively adding a compiler extension when you write a macro.

For transforming data, just use a plain function.

Here is an outline of how you could do it:

(ns tst.demo.core
  (:use demo.core tupelo.core tupelo.test)
  (:require
    [clojure.tools.reader.edn :as edn] ))

(def data {:Location "US-NY-Location1"
           :Priority 3})

(def qual "(Location = \"US\")")

(dotest
  (let-spy [
        ast       (spyx (edn/read-string qual))
        ident-str (first ast)
        ident-kw  (keyword ident-str)
        op        (second ast)
        data-val  (last ast)
        expr      (list op (list ident-kw data) data-val)
        result (eval expr)
        ] 
    ))

and the results:

----------------------------------
   Clojure 1.9.0    Java 10.0.1
----------------------------------

(edn/read-string qual) => (Location = "US")
ast => (Location = "US")
ident-str => Location
ident-kw => :Location
op => =
data-val => "US"
expr => (= (:Location {:Location "US-NY-Location1", :Priority 3}) "US")
result => false

Notice that you still need to fix the "US" part of the location before it will give you a true result.

Docs for let-spy are here and here.

---

Update

For nested expressions, you generally want to use postwalk.

And, don't forget the Clojure CheatSheet!

Answer 2

Here's an example using Instaparse to define a grammar for the criteria and parse string inputs into a syntax tree:

(def expr-parser
  (p/parser
    "<S> = SIMPLE | COMPLEX
     SIMPLE = <'('> NAME <' '> OP <' '> VAL <')'>
     COMPLEX = <'('> S <' '> BOOLOP <' '> S <')'>
     <BOOLOP> = 'or' | 'and'
     NAME = #'[A-Za-z]+'
     VAL = #'[0-9]+' | #'\".+?\"' | 'true' | 'false'
     OP = '=' | '>'"))

And a function to parse and then translate parts of the parsed tree, for easier evaluation later:

(defn parse [s]
  (pt/transform
    {:NAME keyword
     :OP   (comp resolve symbol)
     :VAL  edn/read-string}
    (expr-parser s)))

Some example outputs:

(parse "(Location = \"US\")")
=> ([:SIMPLE :Location #'clojure.core/= "US"])
(parse "(((Location = \"US\") or (Priority > 2)) and (Active = true))")
=>
([:COMPLEX
  [:COMPLEX [:SIMPLE :Location #'clojure.core/= "US"] "or" [:SIMPLE :Priority #'clojure.core/> 2]]
  "and"
  [:SIMPLE :Active #'clojure.core/= true]])

Then a function to evaluate the criteria against a map, without using eval:

(defn evaluate [m expr]
  (clojure.walk/postwalk
    (fn [v]
      (cond
        (and (coll? v) (= :SIMPLE (first v)))
        (let [[_ k f c] v]
          (f (get m k) c))

        (and (coll? v) (= :COMPLEX (first v)))
        (let [[_ lhs op rhs] v]
          (case op
            "or" (or lhs rhs)
            "and" (and lhs rhs)))

        :else v))
    (parse expr)))

(evaluate {:location "US"} "(location = \"US\")")
=> (true)

It also works for nested expressions:

(evaluate
  {:distance 1 :location "MS"}
  "((distance > 0) and ((location = \"US\") or ((distance = 1) and (location = \"MS\"))))")
=> (true)

How can I make this more robust and be more graceful in case of an invalid qual string.

An added benefit of using Instaparse (or similar) is error reporting for "free". Instaparse's errors will pretty print in the REPL, but they can also be treated as maps containing failure specifics.

(defn parse [s]
  (let [parsed (expr-parser s)]
    (or (p/get-failure parsed) ;; check for failure
        (pt/transform
          {:NAME keyword
           :OP   (comp resolve symbol)
           :VAL  edn/read-string}
          parsed))))

(parse "(distance > 2") ;; missing closing paren
=> Parse error at line 1, column 14:
(distance > 2
             ^
Expected:
")" (followed by end-of-string)

Overall, this approach should be safer than eval-ing arbitrary inputs, as long as your parser grammar is relatively limited.