0

Echoing without quotes... 1 line. Fine.

$ echo $(ls -1dmb /bin/*) > test
$ wc -l test
1 test

Echoing with quotes... 396 lines. Bad.

$ echo "$(ls -1dmb /bin/*)" > test
$ wc -l test
396 test

The problem comes when using echo for writing a file and expanding a long variable.

Why does this happen? How to fix it?

Yajo
  • 5,808
  • 2
  • 30
  • 34
  • 3
    That echo is entirely pointless by the way. – Etan Reisner Sep 05 '14 at 13:11
  • Obviously, but it's just an example. The same happens for a very long variable, but the example would be huge. – Yajo Sep 09 '14 at 07:54
  • 1
    Actually, the same thing *doesn't* happen for a long variable. echoing a long variable value will not truncate lines to 80 characters, `ls` is doing that itself. echoing a long variable value *will*, however, lose leading, trailing, and embedded spaces and newlines as that is the shell word splitting after variable expansion and command substitution. – Etan Reisner Sep 09 '14 at 11:16
  • Well, you are right. That happens if you construct your variable hand-written, which is something I also don't understand. But I constructed my variable with `ls` and it got truncated. What I was trying to do is automate the writing of a config file, writing directories separated with comma, but you did not need to know that to answer the question ;). – Yajo Sep 10 '14 at 09:38
  • Like I said, `ls` is doing the line wrapping. The shell is doing the newline (and extra space) dropping. Compare the output of `echo " foo "` to the output of `echo foo ` and you'll see the shell doing what the shell does with whitespace. Do that same thing with a quoted string more than 80 characters long and you **will not** see it wrap in your output (assuming you send it to `less` or a file where you can see that it didn't wrap correctly). – Etan Reisner Sep 10 '14 at 12:38

2 Answers2

2

ls is detecting that your stdout is not a terminal.

check the output of ls -1dmb /bin/* | cat vs ls -1dmb /bin/*. It's ls, who is splitting the output.

Similarly, for ls --color=auto case, color option is used, based on whether the stdout is terminal or not.

When quotes are used, echo is provided with a single arguments, which has embedded newlines, spaces, which are echoed as-is to file.

When quotes are not used, echo is provided multiple arguments, which are split by the IFS. Thus echo prints all of them in a single line. But, don't skip these quotes...

How to fix it:

I think, the splitting always occurs at the end of some file name & never in between a filename. So one of these 2 options may work for you:

ls -1dmb /bin/* | tr '\n' ' ' >test
ls -1dmb /bin/* | tr -d '\n' >test
anishsane
  • 20,270
  • 5
  • 40
  • 73
0

@anishsane correctly answers the topic question (that ls is doing the wrapping and ways to remove them) and covers the quoting issue as well but the quoting issue is responsible for the line count difference and not ls.

The issue here is entirely one of quoting and how command lines, echo, and command substitution all work.

The output from "$(ls ...)" is a single string with embedded newlines protected from the shell via the quotes. Hand that value to echo and echo spits it back out literally (with the newlines).

The output from $(ls ...) is a string that is unprotected from the shell and thus undergoes word splitting and whitespace normalization. Command substitution cannot terminate your command line early (you wouldn't want echo $(ls -1) in a directory with two files to run echo first_file; second_file would you?) the newlines are left as word separators between the arguments to echo. The shell then word splits the result on whitespace (including newlines) and gives echo a list of arguments at which point echo happily executes echo first_file second_file ... which, as you can guess, only outputs a single line of output.

Try this to see what I mean:

$ c() {
    printf 'argc: %s\n' "$#";
    printf 'argv: %s\n' "$@"
}
$ ls *
a.sh b.sh temp
$ ls -1dmb *
a.sh, b.sh, temp
$ c "$(ls -1dmb *)"
argc: 1
argv: a.sh, b.sh, temp
$ c $(ls -1dmb *)
argc: 3
argv: a.sh,
argv: b.sh,
argv: temp
Etan Reisner
  • 77,877
  • 8
  • 106
  • 148
  • `but the quoting issue is responsible for the line count difference and not ls`: Well, I see it in a different way: As we know, [we prefer quotes](http://mywiki.wooledge.org/BashGuide/Practices#Quoting). `ls` is splitting the output into multiple lines. The deviation from the _expected output_ starts from `ls` stage of data flow, rather than `echo "..."` stage. (This is my _opinion_...) – anishsane Sep 05 '14 at 14:15
  • @anishsane `ls` is not creating one line of output, it is creating *n* lines of output (as many as are necessary to list all the files given that it is also wrapping the lines). Shell word splitting/`IFS` and the behaviour of `echo` is what makes the unquoted output one line. The `-1` argument to `ls` here is entirely misleading as it isn't doing anything even when output is going to the terminal. – Etan Reisner Sep 05 '14 at 14:32
  • ^^ Agreed. Also I agree that your comment above answers OP's question, about "why quoting shows the issue & not-quoting solves the issue." What I am saying is that the issue is already introduced by `ls`. It gets _exposed_ by quoting. When not quoted, the issue remains hidden. & yes, `-1` is indeed misleading. – anishsane Sep 05 '14 at 15:04
  • @anishsane The issue in the post topic is introduced by `ls`, yeah. The issue in the body is not `ls` but `echo` and quoting. The real problem here is that the topic and the body don't agree on what question they are asking. =) Also the confusion about `-1` being meaningful isn't helping the OP at all. – Etan Reisner Sep 05 '14 at 15:11