5

The POSIX spec states with regard to Arithmetic Expansion that

[i]f the shell variable x contains a value that forms a valid integer constant, optionally including a leading plus or minus sign, then the arithmetic expansions "$((x))" and "$(($x))" shall return the same value.

Which is a reasonable shortcut and cleans up complicated expressions rather nicely.

bash (versions 3.2.25(1)-release from CentOS 5 and 4.3.33(1)-release from debian unstable) as well as ksh (Version AJM 93t+ 2010-06-21 from CentOS 5) all seem to go one step farther then that however.

They all seem to recursively expand variables encountered in arithmetic expansion (and numeric contexts in [[ resulting from using the numeric operators).

Specifically:

$ set -x
$ bar=5
+ bar=5
$ foo=bar
+ foo=bar
$ [ foo -gt 4 ]; echo $?
+ '[' foo -gt 4 ']'
-bash: [: foo: integer expression expected
+ echo 2
2
$ [[ foo -gt 4 ]]; echo $?
+ [[ foo -gt 4 ]]
+ echo 0
0
$ [[ foo -eq 0 ]]; echo $?
+ [[ foo -eq 0 ]]
+ echo 1
1
$ [[ foo -eq 5 ]]; echo $?
+ [[ foo -eq 5 ]]
+ echo 0
0
$ (( foo == bar )); echo $?
+ ((  foo == bar  ))
+ echo 0
0
$ (( foo == 1 )); echo $?
+ ((  foo == 1  ))
+ echo 1
1

Where does this behavior come from and why would it ever be desirable?

It makes using [[ in place of [ explicitly less safe when used with numeric operators because invalid values and typographical errors go from being script errors to being silently valid (but likely erroneous) tests.

Edit: As a side question if anyone happens to know when this "feature" was added to bash/etc. I would be interested in knowing that as well.

Etan Reisner
  • 77,877
  • 8
  • 106
  • 148
  • You don't need all those examples, a simple `echo $((foo))` shows the behavior. – Barmar May 01 '15 at 03:05
  • Why are all the statements in your example script commented out? How are they executing anyway? – Barmar May 01 '15 at 03:09
  • @Barmar `set -x`, then execute the commented commands. Uncommented lines are outputs. I have to say that's rather confusing. Should have been the other way round. – 4ae1e1 May 01 '15 at 03:11
  • Is `#` the shell prompt from an interactive session? You're doing this as root? – Barmar May 01 '15 at 03:12
  • Hmm, interesting... I'd recommend normalizing the prompt to `> ` or `$ ` anyway when you post it. – 4ae1e1 May 01 '15 at 03:12
  • I usually do sanitize to `$` for my prompts. I guess I got distracted this time. – Etan Reisner May 01 '15 at 03:17
  • No explanation for this, but just for the record `set -o posix` doesn't help here. – 4ae1e1 May 01 '15 at 03:24

1 Answers1

3

It's worse than you think. The value of the variable is recursively treated as an arithmetic expression:

$ foo='bar+bar'
$ echo $((foo))
10

The bash manual section on Shell Arithmetic says:

The value of a variable is evaluated as an arithmetic expression when it is referenced, or when a variable which has been given the integer attribute using ‘declare -i’ is assigned a value.

The latter part means you can do:

$ declare -i int
$ int=bar+bar
$ echo $int
10

Note that none of this is a violation of the spec you quoted above. It only says what should be done if the value of the variable is an integer constant. It doesn't say what should be done if the value is something else, which leaves that open for implementations to add extensions like this. Bash Hackers Wiki explains it:

If the variable doesn't hold a value that looks like a valid expression (numbers or operations), the expression is re-used to reference, for example, the named parameters

If the eventual expansion is not a valid expression, you'll get an error:

$ foo='a b'
$ bar=foo
echo $((bar))
bash: a b: syntax error in expression (error token is "b")

So if your variable contains random stuff, it's likely to cause an error. But if it just contains a single word, which is valid variable syntax, that will evaluate as 0 if the variable isn't set.

Barmar
  • 741,623
  • 53
  • 500
  • 612
  • I can understand the `declare -i` bit actually. That's surprising but somewhat elegant (though I can't imagine using that as it **will** cause someone to make a mistake at some point I'm sure). I can see the argument for the statement evaluation actually "I want to build up a complex arithmetic statement and then use it later". I actually think that may be **less** troubling than what I started with (at least assuming it stopped at one level of expansion which I'm sure it doesn't). – Etan Reisner May 01 '15 at 03:22
  • No, it keeps expanding until it gets either a number or something that isn't a valid expression. – Barmar May 01 '15 at 03:25
  • Try setting the final variable to `1/`, which is not a valid expression. – Barmar May 01 '15 at 03:33
  • Ah. Fair enough. Not any more helpful in general but fair enough. – Etan Reisner May 01 '15 at 03:33
  • Also try setting it to `"a b"`. Variables can't have spaces in their names, so that's an invalid expression. – Barmar May 01 '15 at 03:35
  • I think I would probably have argued that the spec only indicates that bare variable names are valid when they expand to a valid integer constant and so this extension violates the spec but I can see how that's not a necessary interpretation. – Etan Reisner May 01 '15 at 03:36
  • As a general rule, when standards don't say you have to detect something as an error, it's unspecified, and implementations are free to define the meaning. This is how they typically allow extensions. – Barmar May 01 '15 at 03:39
  • 2
    ... and shells are particularly happy to exploit unspecified syntagma :) – rici May 01 '15 at 03:41
  • Oof, how about this: `foo=bar; bar=5; : $((foo++))`. `foo` is dereferenced to `bar`, but `bar` is not modified, only `foo`. – chepner Sep 01 '15 at 04:29
  • @chepner `$((foo++)` is short for `foo=$((foo+1))`, so it only needs to dereference `foo` in the right side of the assignment. – Barmar Sep 01 '15 at 05:06