0

Walking a directory tree in Emacs using the cookbook recipe (http://www.emacswiki.org/emacs/ElispCookbook#toc59), or the solution at Walk up the directory tree is quite slow.

Could one use Unix's find instead, via shell-command or call-process, and perform a funcall on the returned list?

Is there any cons to that idea (perhaps too much memory consumption for large trees?), and what would be the idiomatic way to do that in elisp, ie calling find with some given arguments and mapping a funcall on the returned value?

One possible benefit I can see is that the shell process could be launched asynchronously, without Emacs stopping at all when the process is started.

Community
  • 1
  • 1
gsl
  • 1,063
  • 1
  • 16
  • 27

1 Answers1

1

Yes, of course you can call find via call-process an then split the result line-by-line. Note that walk-path can be made significantly more efficient by the use of file-name-all-completions in place of directory-files and file-directory-p:

(defun my-walk-directory (dir action)
  "walk DIR executing ACTION with (dir file).
DIR needs to end with /."
  (dolist (file (let ((completion-ignored-extensions nil)
                      (completion-regexp-list nil))
                  (file-name-all-completions "" dir)))
    (cond
     ((member file '("./" "../")) nil)
     ((eq ?/ (aref file (1- (length file)))) (my-walk-directory (concat dir file) action))
     (t (funcall action dir file)))))

Of course, it's still not going to be as fast as find, but in my experience, this is about 10 times faster than using directory-files plus file-directory-p.

Stefan
  • 27,908
  • 4
  • 53
  • 82
  • Wouldn't unfolding the recursion with an explicit directory stack make the function more efficient as well? I though function calls were rather slow in Emacs Lisp. –  Sep 13 '14 at 15:07
  • I don't think the difference would be measurable. Feel free to try it, of course. – Stefan Sep 13 '14 at 15:09
  • @Stefan, @lunaryorn Thank you for the answer and the extra information. I would like to try to use the Unix option as well, and compare the speeds. Could you give me a pointer on how to convert the value returned by `call-process` to a list, or how to split the result line-by-line, as you were saying? – gsl Sep 13 '14 at 15:49
  • `call-process` can put the result in the current buffer, so you can then use `search-forward` to look for the next `\n` and then `buffer-substring` to extract the line you just found. – Stefan Sep 13 '14 at 18:20
  • @Stefan, thank you so much. Just as a reference, I have found a recent article about speeding up file find. http://www.unixlore.net/articles/speeding-up-bulk-file-operations.html Interestingly, the last Perl solution is faster than running two xargs commands in a row. Of course, gnu `locate` with a fresh db should be by far the fastest. – gsl Sep 13 '14 at 19:09