Handle special characters in bash for...in loop

Question

Suppose I've got a list of files

file1
"file 1"
file2

a for...in loop breaks it up between whitespace, not newlines:

for x in $( ls ); do
   echo $x
done

results:

file
1
file1
file2

I want to execute a command on each file. "file" and "1" above are not actual files. How can I do that if the filenames contains things like spaces or commas?

It's a little trickier than I think find -print0 | xargs -0 could handle, because I actually want the command to be something like "convert input/file1.jpg .... output/file1.jpg" so I need to permutate the filename in the process.

score 11 · Answer 1 · answered Oct 14 '09 at 02:38

11

Actually, Mark's suggestion works fine without even doing anything to the internal field separator. The problem is running ls in a subshell, whether by backticks or $( ) causes the for loop to be unable to distinguish between spaces in names. Simply using

for f in *

instead of the ls solves the problem.

#!/bin/bash
for f in *
do
 echo "$f"
done

answered Oct 14 '09 at 02:38

Jordan

237
1
5

Jordan, I didn't see any problems with eduffy's solution. I tried with files that had spaces in them. In fact, ls -1 doesn't create a sub shell. It just creates another process, that's all. – codeforester Dec 31 '16 at 01:38
I agree, it works fine for me. Honestly, given when I made this comment and the fact that there's an answer I refer to from "Mark" that's missing, I think some context has been lost that might make more sense. @eduffy's solution is just fine, the extra process isn't a huge deal though if automating over a very large set of locations, /maybe/ it would be slightly slower, but I doubt it would ever be a problem in the real world. – Jordan Jan 10 '17 at 20:57

eduffy · Accepted Answer · 2016-02-18T16:41:14.780

2

UPDATE BY OP: this answer sucks and shouldn't be on top ... @Jordan's post below should be the accepted answer.

~~one possible way:~~

ls -1 | while read x; do
   echo $x
done

edited Feb 18 '16 at 16:41

answered Oct 14 '09 at 01:47

eduffy

39,140
13
95
92

@ʞɔıu, if the answer is wrong, then you can un-accept it so that it won't mislead anybody. – codeforester Jan 11 '17 at 00:06
@codeforester - I made the edit saying the answer sucked, not ʞɔıu. – eduffy Jan 11 '17 at 01:13

score 0 · Answer 3 · answered Jan 20 '10 at 23:12

I know this one is LONG past "answered", and with all due respect to eduffy, I came up with a better way and I thought I'd share it.

What's "wrong" with eduffy's answer isn't that it's wrong, but that it imposes what for me is a painful limitation: there's an implied creation of a subshell when the output of the ls is piped and this means that variables set inside the loop are lost after the loop exits. Thus, if you want to write some more sophisticated code, you have a pain in the buttocks to deal with.

My solution was to take the "readline" function and write a program out of it in which you can specify any specific line number that you may want that results from any given function call. ... As a simple example, starting with eduffy's:

ls_output=$(ls -1)
# The cut at the end of the following line removes any trailing new line character
declare -i line_count=$(echo "$ls_output" | wc -l | cut -d ' ' -f 1)
declare -i cur_line=1 
while [ $cur_line -le $line_count ] ;
do  
   # NONE of the values in the variables inside this do loop are trapped here.
   filename=$(echo "$ls_output" | readline -n $cur_line)
   # Now line contains a filename from the preceeding ls command
   cur_line=cur_line+1
done

Now you have wrapped up all the subshell activity into neat little contained packages and can go about your shell coding without having to worry about the scope of your variable values getting trapped in subshells.

I wrote my version of readline in gnuc if anyone wants a copy, it's a little big to post here, but maybe we can find a way...

Hope this helps, RT

Handle special characters in bash for...in loop

3 Answers3

Linked