Recursively list hidden files without ls, find or extendedglob

Question

As an exercise I have set myself the task of recursively listing files using bash builtins. I particularly don't want to use ls or find and I would prefer not to use setopt extendedglob. The following appears to work but I cannot see how to extend it with /.* to list hidden files. Is there a simple workaround?

g() { for k in "$1"/*; do # loop through directory
[[ -f "$k" ]] && { echo "$k"; continue; }; # echo file path
[[ -d "$k" ]] && { [[ -L "$k" ]] && { echo "$k"; continue; }; # echo symlinks but don't follow
g "$k"; }; # start over with new directory
done; }; g "/Users/neville/Desktop" # original directory

Added later: sorry - I should have said: 'bash-3.2 on OS X'

Starting in `bash` 4.1, you can use extended patterns in the argument to `==` and `!=` without explicitly turning on `extglob`, if that's your objection. — chepner, Jan 18 '16 at 20:45

rici · Answer 1 · 2016-01-18T20:36:02.663

Change

for k in "$1"/*; do

to

for k in "$1"/* "$1"/.[^.]* "$1"/..?*; do

The second glob matches all files whose names start with a dot followed by anything other than a dot, while the third matches all files whose names start with two dots followed by something. Between the two of them, they will match all hidden files other than the entries . and ...

Unfortunately, unless the shell option nullglob is set, those (like the first glob) could remain as-is if there are no files whose names match (extremely likely in the case of the third one) so it is necessary to verify that the name is actually a file.

An alternative would be to use the much simpler glob "$1"/.*, which will always match the . and .. directory entries, and will consequently always be substituted. In that case, it's necessary to remove the two entries from the list:

for k in "$1"/* "$1"/.*; do
  if ! [[ $k =~ /\.\.?$ ]]; then
    # ...
  fi
done

(It is still possible for "$1"/* to remain in the list, though. So that doesn't help as much as it might.)

I actually like this answer better than mine (and this is how I would personally do it) because it works in shells other than Bash. But since the OP did say "bash" specifically, I went ahead and contributed the GLOBIGNORE solution, which automagically skips `.` and `..`. :) — dannysauer, Jan 18 '16 at 22:02
Both of these work well and I can see what they are doing but I don't understand why. Without them use of "$1"/.* slowly produces many directories: /Users/neville/Desktop/test dir/././././././././././. - Why is this? Is the dot file acting like a symlink to a directory and if so why is it not stopped by my symlink trap? If I was happy to list all the UNIX dot and double dot files how should it be modified? — Neville Hillyer, Jan 21 '16 at 14:52

score 1 · Answer 2 · answered Jan 18 '16 at 21:57

Set the GLOBIGNORE file to exclude . and .., which implicitly turns on "shopt -u dotglob". Then your original code works with no other changes.

user@host [/home/user/dir]
$ touch file
user@host [/home/user/dir]
$ touch .dotfile
user@host [/home/user/dir]
$ echo *
file
user@host [/home/user/dir]
$ GLOBIGNORE=".:.."
user@host [/home/user/dir]
$ echo *
.dotfile file

Note that this is bash-specific. In particular, it does not work in ksh.

Score_Under · Answer 3 · 2016-01-18T20:59:17.890

You can specify multiple arguments to for:

for k in "$1"/* "$1"/.*; do

But if you do search for .* in directories , you should be aware that it also gives you the . and .. files. You may also be given a nonexistent file if the "$1"/* glob matches, so I would check that too.

With that in mind, this is how I would correct the loop:

g() {
    local k subdir
    for k in "$1"/* "$1"/.*; do # loop through directory
        [[ -e "$k" ]] || continue  # Skip missing files (unmatched globs)
        subdir=${k##*/}
        [[ "$subdir" = . ]] || [[ "$subdir" = .. ]] && continue  # Skip the pseudo-directories "." and ".."

        if [[ -f "$k" ]] || [[ -L "$k" ]]; then
            printf %s\\n "$k"  # Echo the paths of files and symlinks
        elif [[ -d "$k" ]]; then
            g "$k"  # start over with new directory
        fi
    done
}
g ~neville/Desktop

Here the funky-looking ${k##*/} is just a fast way to take the basename of the file, while local was put in so that the variables don't modify any existing variables in the shell.

One more thing I've changed is echo "$k" to printf %s\\n "$k", because echo is irredeemably flawed in its argument handling and should be avoided for the purpose of echoing an unknown variable. (See Rich's sh tricks for an explanation of how; it boils down to -n and -e throwing a spanner in the works.)

By the way, this will NOT print sockets or fifos - is that intentional?

Recursively list hidden files without ls, find or extendedglob

3 Answers3