How to rename a clojure file and namespace

Question

I've actually got the renaming the file portion down, but now I'm trying to figure out how to rename a namespace. The idea here is I want to iterate through a bunch of .clj files in a directory and rename them. Ex:

File: some-file-123.clj contents:

(ns some-file-123
  (require [clojure.string :as str]))

(defn some-func [] (println "ima func"))

->

File: 123-some-file.clj contents:

(ns 123-some-file
  (require [clojure.string :as str]))

(defn some-func [] (println "ima func"))

I've got all of the string manipulation stuff under control, what I'm trying to understand is how to go about reading clojure file contents, identifying the namespace, and replacing it.

What I've come across so far involves using clojure.walk, for example:

(let [file (first (get-some-file-fn))
      file-text (read-string (slurp file))]
    (clojure.walk/postwalk-demo file-text))

I know I can use clojure.walk/postwalk-replace to do some replacement, but I'm stuck here.

How do I

Extract the namespace from the file-text
Replace the namespace with some arbitrary new name using postwalk-replace
Is this even the right approach?

Just a note that if the namespace has - in it, the filename should have _ in the corresponding positions (so some_file_123.clj would have ns some-file-123 in it). — Sean Corfield, Oct 12 '19 at 23:22
could you please add some context: is your goal to rename the file from inside the project, or from outside? maybe you want to do it from repl? or external script? I mean, do you have an access to the renamed file's project's classpath ? — leetwinski, Oct 14 '19 at 09:05

Alan Thompson · Answer 1 · 2019-10-13T23:35:22.530

Here is an example of how to read Clojure source code and manipulate it using the tupelo.forest library. You can see the live code in the GitHub repo.

First set up a test and parse the source:

(dotest
  (hid-count-reset)
  (with-forest (new-forest)
    (let [debug-flg       true
          edn-str ;   Notice that there are 3 forms in the source
                          (ts/quotes->double
                            "(ns tst.demo.core
                               (:use demo.core tupelo.core tupelo.test))

                             (defn add2 [x y] (+ x y))

                             (dotest
                               (is= 5 (spyx (add2 2 3)))
                               (is= 'abc' (str 'ab' 'c'))) ")

          ; since `edn/read-string` only returns the next form,
          ; we wrap all forms in an artifical [:root ...] node
          parse-txt       (str "[:root " edn-str " ]")
          edn-data        (edn/read-string parse-txt) ; reads only the first form

At this point, we load the EDN data into a tree using the tupelo.forest lib:

          root-hid        (add-tree-edn edn-data) ; add edn data to a single forest tree
          ns-path         (only (find-paths root-hid [:** {::tf/value (symbol "ns")}])) ; search for the `ns` symbol`
          ; ns-path looks like  `[1038 1009 1002]`, where 1002 points to the `ns` node

          ns-hid          (xlast ns-path) ; ns-hid is a pointer to the node with `ns`
          ns-parent-hid   (xsecond (reverse ns-path)) ; get the parent hid (eg 1009)
          ns-parent-khids (hid->kids ns-parent-hid) ; vector with `ns` contains 4 kids, of which `ns` is the first
          ns-sym-hid      (xsecond ns-parent-khids)] ; symbol `tst.demo.core` is the 2nd kid
      (when debug-flg
        (newline)
        (spyx-pretty (hid->bush root-hid))
        (newline)
        (spyx (hid->node ns-hid))
        (spyx (hid->node ns-parent-hid))
        (spyx ns-parent-khids)
        (newline)
        (spyx (hid->node ns-sym-hid)))

The above debug printouts show what is happening. Here is a "bush" view of the tree structure:

(hid->bush root-hid) => 
[{:tag :tupelo.forest/vec, :tupelo.forest/index nil}
 [#:tupelo.forest{:value :root, :index 0}]
 [{:tag :tupelo.forest/list, :tupelo.forest/index 1}
  [#:tupelo.forest{:value ns, :index 0}]
  [#:tupelo.forest{:value tst.demo.core, :index 1}]
  [{:tag :tupelo.forest/list, :tupelo.forest/index 2}
   [#:tupelo.forest{:value :use, :index 0}]
   [#:tupelo.forest{:value demo.core, :index 1}]
   [#:tupelo.forest{:value tupelo.core, :index 2}]
   [#:tupelo.forest{:value tupelo.test, :index 3}]]]
 [{:tag :tupelo.forest/list, :tupelo.forest/index 2}
  [#:tupelo.forest{:value defn, :index 0}]
  [#:tupelo.forest{:value add2, :index 1}]
  [{:tag :tupelo.forest/vec, :tupelo.forest/index 2}
   [#:tupelo.forest{:value x, :index 0}]
   [#:tupelo.forest{:value y, :index 1}]]
  [{:tag :tupelo.forest/list, :tupelo.forest/index 3}
   [#:tupelo.forest{:value +, :index 0}]
   [#:tupelo.forest{:value x, :index 1}]
   [#:tupelo.forest{:value y, :index 2}]]]
 [{:tag :tupelo.forest/list, :tupelo.forest/index 3}
  [#:tupelo.forest{:value dotest, :index 0}]
  [{:tag :tupelo.forest/list, :tupelo.forest/index 1}
   [#:tupelo.forest{:value is=, :index 0}]
   [#:tupelo.forest{:value 5, :index 1}]
   [{:tag :tupelo.forest/list, :tupelo.forest/index 2}
    [#:tupelo.forest{:value spyx, :index 0}]
    [{:tag :tupelo.forest/list, :tupelo.forest/index 1}
     [#:tupelo.forest{:value add2, :index 0}]
     [#:tupelo.forest{:value 2, :index 1}]
     [#:tupelo.forest{:value 3, :index 2}]]]]
  [{:tag :tupelo.forest/list, :tupelo.forest/index 2}
   [#:tupelo.forest{:value is=, :index 0}]
   [#:tupelo.forest{:value "abc", :index 1}]
   [{:tag :tupelo.forest/list, :tupelo.forest/index 2}
    [#:tupelo.forest{:value str, :index 0}]
    [#:tupelo.forest{:value "ab", :index 1}]
    [#:tupelo.forest{:value "c", :index 2}]]]]]

and the other debug printouts:

(hid->node ns-hid) => #:tupelo.forest{:khids [], :value ns, :index 0}
(hid->node ns-parent-hid) => {:tupelo.forest/khids [1002 1003 1008], :tag :tupelo.forest/list, :tupelo.forest/index 1}
ns-parent-khids => [1002 1003 1008]

(hid->node ns-sym-hid) => #:tupelo.forest{:khids [], :value tst.demo.core, :index 1}

We then replace the old namespace symbol & convert the forms back to string format:

      ; replace the old namespace symbol with a new one
      (attrs-merge ns-sym-hid {::tf/value (symbol "something.new.core")})

      ; find the 3 kids of the `:root` node
      (let [root-khids      (it-> root-hid
                              (hid->node it)
                              (grab ::tf/khids it)
                              (drop 1 it) ;  remove :root tag we added
                              )
            kids-edn        (forv [hid root-khids] ; still 3 forms to output
                              (hid->edn hid))
            modified-src    (with-out-str ; convert EDN forms to a single string
                              (doseq [form kids-edn]
                                (prn form)))
            ; expected-result is the original edn-str but with the new namespace symbol
            expected-result (str/replace edn-str "tst.demo.core" "something.new.core")]

        (when debug-flg
          (spyx (hid->node ns-sym-hid))
          (newline)
          (spyx-pretty kids-edn)
          (newline)
          (println :modified-src \newline modified-src))

And the debug printouts show it in action:

(hid->node ns-sym-hid) => #:tupelo.forest{:khids [], :value tst.demo.core, :index 1}
(hid->node ns-sym-hid) => #:tupelo.forest{:khids [], :value something.new.core, :index 1}

kids-edn => 
[(ns something.new.core (:use demo.core tupelo.core tupelo.test))
 (defn add2 [x y] (+ x y))
 (dotest (is= 5 (spyx (add2 2 3))) (is= "abc" (str "ab" "c")))]

:modified-src 
(ns something.new.core (:use demo.core tupelo.core tupelo.test))
(defn add2 [x y] (+ x y))
(dotest (is= 5 (spyx (add2 2 3))) (is= "abc" (str "ab" "c")))

The single unit test verifies the modified source is as expected (ignoring whitespace):

        (is-nonblank= modified-src expected-result)))))

leetwinski · Answer 2 · 2019-10-14T10:28:15.750

if you are using emacs for clojure development, you could be aware of the cool clj-refactor

Under the hood it uses the refactor-nrepl clojure library. So you can employ it's api for programmatic refactoring.

first of all add the dependency to your project.clj:

:dependencies [[org.clojure/clojure "1.10.0"]
               [refactor-nrepl "2.4.0"]
              ;; rest deps
              ]

then, you can use the rename-file-or-dir function for file renaming: the only trouble, is that this function operates on files, based on the classpaths, so to make refactoring on arbitrary path you need to do some plumbing:

that is what it's like for my unrelated sample project:

(require '[refactor-nrepl.rename-file-or-dir :as r])

;; temporarily redefining the lookup paths for source code 
;; (unfortunately there is no setting in the library for that
(with-redefs [refactor-nrepl.core/dirs-on-classpath (constantly
                                                     (list (java.io.File. "/home/leetwin/dev/projects/clojure/ooo/src")
                                                           (java.io.File. "/home/leetwin/dev/projects/clojure/ooo/test")))]
  (r/rename-file-or-dir
   "/home/leetwin/dev/projects/clojure/ooo/src/playground/core.clj"
   "/home/leetwin/dev/projects/clojure/ooo/src/playground/hardcore.clj"))

;;=> ("/home/leetwin/dev/projects/clojure/ooo/test/playground/core_test.clj"
;;    "/home/leetwin/dev/projects/clojure/ooo/src/playground/hardcore.clj")

you provide the source roots of your project (in my case ...src/ and ...test/) and the absolute path of of the file/dir to be ranamed, and the lib does the rest.

Notice, that the refactoring affects 2 files: the source file itself, and the test file which has the reference to it's namespace. The file is renamed, and all the playground.core refs become playground.hardcore (refactor-nrepl tries to replace the needed references everywhere in classpath provided)

it can also rename the whole directory, rewriting the references inside all the inner files (and related ones fro mother dirs of course):

(r/rename-file-or-dir
   "/home/leetwin/dev/projects/clojure/ooo/src/playground"
   "/home/leetwin/dev/projects/clojure/ooo/src/underground")

I would say it is a better way to do your task, since it is quite a widely used library, which aims to be THE refactoring tool for clojure.

Otherwise,clojure-walk replacing does the trick to some extent (and absolutely ok for fun and education)

score 0 · Answer 3 · answered Oct 29 '19 at 02:12

For an approach without libraries, see my answer to another question.

In short

(with-open [reader (java.io.PushbackReader. 
                     (clojure.java.io/reader "src/my/file/ns_path.clj"))]
  (loop [[ forms done? ] 
         [ []    false ]]
    (if done?
      forms
      (recur (try
               [(conj forms (read reader)) false]
               (catch Exception ex
                 (println (.getMessage ex))
                 [forms true]))))))

Will get you the forms from a file as a vector, which you can then transform however you'd like before writing them back to the file. For that, you could try something like

(run! #(spit "my/source/file/path.clj"
         (with-out-str (clojure.pprint/pprint %))
         :append true)
  my-transformed-forms)

with-out-str captures the output that pprint would normally stream to *out* and returns it as a string.

How to rename a clojure file and namespace

3 Answers3