Is a newline in a filename still problematic when using ls

Question

As mentioned here:

https://stackoverflow.com/a/5886002/2534715
http://mywiki.wooledge.org/ParsingLs (last section)

newlines \n in filenames are an issue, when pipe-lining the output of ls to another command, because the newline will split one filename into two lines.

The man page of ls states:

-b, --escape                print C-style escapes for nongraphic characters
-q, --hide-control-chars    print ? instead of nongraphic characters
-1                          list one file per line.  Avoid '\n' with -q or -b

So my interpretation is, using the -b or -q options resolves this issue. Or are there still cases left, which are not covered by this approach?

score 3 · Accepted Answer · answered Oct 13 '18 at 13:50

How will the next command you pipe the output to deal with those escapes? How do you find files deeper in the directory tree? Is it potable to run on POSIX that isn't Linux? The GNU version of ls does some filtering by default, but that's not on every system.

David Wheeler's essay Fixing Unix/Linux/POSIX Filenames: Control Characters (such as Newline), Leading Dashes, and Other Problems goes over some of the problems. Lots of software assumes various control characters like \n or \t can be used as delimiters, so files with those can be problematic. However, there are ways to deal with file names containing practically any character:

It’s worth noting that if you want to handle fully-arbitrary filenames, use “find . ... -exec” when you can; that’s 100% portable, and can handle arbitrarily-awkward filenames. The more-recent POSIX addition to find of -exec ... {} + can help too. So where you can, do this kind of thing:
 # This is correct and portable; painful if "command" gets long:
 find . ... -exec command {} ;
 # This is correct and standard; some systems don't implement this:
 find . ... -exec command {} +

If you can prevent files containing \n, you can use the trick of setting a shell input field separator to just say newline and tab. Then use the file name variable in a shell script.

  IFS="`printf '\n\t'`"
  for file in `find . -type f` ; do
    some_command "$file" ...
  done

Is a newline in a filename still problematic when using ls

1 Answers1