How bash IFS is working in this example?

Question

I am reading about bash internal variables and came across this IFS example:

output_args_one_per_line()
{
  for arg
  do
    echo "[$arg]"
  done #  ^    ^   Embed within brackets, for your viewing pleasure.
}

CASE1

IFS=" "
var=" a  b c   "

output_args_one_per_line $var
#OUTPUT
# [a]
# [b]
# [c]

CASE2

IFS=:
var=":a::b:c:::"               # Same pattern as above
CASE2   
                        # but substituting ":" for " "  ...
output_args_one_per_line $var
# []
# [a]
# []
# [b]
# [c]
# []
# []

Now, according to my understanding if value of IFS is default \t\n than leading and trailing whitespaces are removed. So, for case1 bash sees var as a b c therefore the output.

For case2 according to me bash sees var as |a||b|c||| treat | as space here. I checked it using

Noob@Noob:~/tmp$ IFS=$':' FOO=$":a::b:c:::"; echo $FOO $'x'
 a  b c   x

So, my expected output for case 2 is

# []
# [a]
# []
# []
# [b]
# [c]
# []
# []
# []

So, can someone explain to me internally how bash is treating var in case 2 and where am I going wrong iin my understanding.

OT: your output_args_one_per_line function could be replaced by `printf "[%s]\n" $var` — Mark Reed, Oct 17 '12 at 11:45
look at the split as presented with `bash -x` when `var` is expanded. see the edit on my answer. — c00kiemon5ter, Oct 17 '12 at 12:01

score 3 · Answer 1 · answered Oct 17 '12 at 12:46

3

Your claim (edited to use ':' instead of '|'):

For case2 according to me bash sees var as :a::b:c::: treat : as space here.

is false. IFS causes bash to treat : as a word separator, not whitespace. Don't confuse the two just because whitespace is the default word separator.

answered Oct 17 '12 at 12:46

chepner

497,756
71
530
681

You are right I misspoke there what I actually meant is bash changes `a:b` to `a b` using whitespace as a separator. – RanRag Oct 17 '12 at 13:52

score 1 · Answer 2 · answered Oct 17 '12 at 11:41

1

: is the delimiter. a:b is split into a and b, nothing in between. In your expected behaviour, how would you encode the following?

[a]
[]
[b]

The only strange thing is there are not three empty strings at the end. This might be because

Unquoted implicit null arguments, resulting from the expansion of parameters that have no values, are removed.

answered Oct 17 '12 at 11:41

choroba

231,213
25
204
289

If I understand you correctly than `a::b` is treated as `a(space)(empty word)b which will result into `a[]b. Can you please elaborate a little on the `strange thing` because am a bash noob and I don't understand official docs well without an example. – RanRag Oct 17 '12 at 11:49
you split from the beginning to next met `IFS`. Then from that point to the next met `IFS`, etc.. a string like `::` will return `[]` which is _the empty string_ because from the first `:` to the next `:` you have nothing in between, but the split takes place. – c00kiemon5ter Oct 17 '12 at 11:57
@Noob - in your expected output, you claim that `a::b` should have two empty strings between `a` and `b`, when really it should only be one.. – Mark Reed Oct 17 '12 at 11:59
@c00kiemon5ter: Thanks, really helpful explanation. – RanRag Oct 17 '12 at 12:01
1

The removal of the trailing null argument (but not the leading one) comes from the wording of the POSIX standard. It was reportedly hotly debated on the committee, and is an unpopular decision among many of the people who care about that sort of thing. But it is at least consistent across the various shells (ksh, {d,}ash, zsh, etc). It's just one of those quirks you have to account for. – Mark Reed Oct 17 '12 at 12:07

c00kiemon5ter · Accepted Answer · 2012-10-17T12:00:54.517

On case 2 you have ":a::b:c:::" with IFS=:
so what you are doing is splitting the string in each met :

so, from the beginning of the string, till the first : you have ":
which is nothing, thus []

from the first : to the next : you have :a:
which is a, thus [a]

from there to the next : you have ::
which is nothing, thus []

from there to the next : you have :b:
which is b, thus [b]

from there to the next : you have :c:
which is c, thus [c]

from there to the next : you have ::
which is nothing, thus []

from there to the next : you have ::"
which is nothing, thus []

so you get that result..

as @Mark Reed mentioned in the comments you can use printf
along with bash -x you get:

$ bash -x
$ IFS=':'; var=':a::b:c:::'; printf "[%s]\n" $var
[12]+ IFS=:
[12]+ var=:a::b:c:::
[12]+ printf '[%s]\n' '' a '' b c '' ''     # notice this
[]
[a]
[]
[b]
[c]
[]
[]

score 1 · Answer 4 · answered Oct 17 '12 at 15:08

The reason is pretty simple:

IFS=" "
var=" a  b c   "

output_args_one_per_line $var

this means output_args_one_per_line is called with this argument(s):

output_args_one_per_line  a  b c

While parsing the command line, BASH will remove the additional white space, so the actual call will use

output_args_one_per_line a b c

i.e. multiple spaces are merged into one, the spaces before a will become the space between command and the first argument.

That means the spaces will be gone before IFS is applied. It also means you can't write

IFS=:
output_args_one_per_line:$var

There must be a white space after the command, not a word separator.

You can run the script with set -x to see the trace output (i.e. how BASH expands lines).

In the second case, the word delimiter isn't a space character, so the necessary whitespace between command and first argument isn't merged with the arguments and the line becomes

output_args_one_per_line :a::b:c:::

The only odd thing is that the output should be three empty arguments after c but that is probably because empty trailing arguments are removed (just like BASH removes white space after the arguments). Here is another odd output:

IFS=:
var=":a::b:c::: "   # Blank after C
> output_args_one_per_line $var
[]
[a]
[]
[b]
[c]
[]
[]
[ ]

So if var does contain anything after the last colon, we get the missing argument.

How bash IFS is working in this example?

4 Answers4