5

More precisely, why does

"`command "$variable"`"

treat the outer quotes as enclosing the inner quotes, instead of expanding the variable outside any quotes?

The exact command I used to test this is similar to an example brought up in another stackoverflow question about the correct method of quoting when using command substitution:

fileName="some path with/spaces"
echo "`dirname "$fileName"`"

which correctly echoes "some path with", instead of complaining because of an invalid number of arguments.

I read Bash's man page, where it states in chapter "EXPANSION", section "Commmand Substitution" that the new-style $() substitution preserves the meaning of any character between the parentheses, however, regarding backquotes, it only mentions that backslashes work in a limited way:

When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution.

My first thought was that backticks do the same, aside from the mentioned exception, thus "quoting" the inner double quotes, however, I got told that is not the case. The second observation that pointed me to this direction was that

a=\$b
b=hello
echo `echo $a`

prints "$b". Had the backticks let the dollar sign get interpreted, the first variable substitution should have occurred before the subshell was invoked, with the subshell expanding the string "$b", resulting in "hello." According to the above excerpt from the man page, I can even make sure the dollar sign is actually quoted, by using

echo `echo \$a`

and the results would still be the same.

A third observation gives me some doubts though:

echo `echo \\a`

Result: "\a"

echo \a

Result: a

Here it seems like both backslashes were retained until the subshell came into play, even though the man page states that backslashes within backquotes do not have their literal meaning when followed by another backslash. EDIT: ^ Everything works as expected in this regard, I must have used the wrong shell (tcsh in my other terminal, and with a different character from "a").

Although I have not been able to find out what actually happens, while I was searching for the answer, I came across some people mentioning the term "quoting context" with regards to command substitution, but without any explanation as to what it means or where it is described. I have not found any real reference to "quoting contexts" in either Bash references (gnu.org, tldp, man bash) or via DuckDuckGo.

Additionally to knowing what is going on, I'd preferably like to have some reference or guidance as to how this behavior can be discerned from it, because I think I might have failed to put some pieces together from which this naturally comes. Otherwise I'll just forget the answer.

To those recommending people to use the new-style dollar sign and parentheses substitution: on ca. 50 years old Unix machines with tens or hundreds of different proprietary environments (can't throw out a shell for a newer one), when one has to write scripts compatible between most shells that anyone might be using, it is not an option.

Thanks to anyone who can help me.

Larry
  • 427
  • 2
  • 10
  • 4
    Can you clarify this sentence: "treat the outer quotes as enclosing the single quotes, instead of expanding the variable outside any quotes?" This page http://mywiki.wooledge.org/BashFAQ/082 might be helpful to understand the way backticks behave. For example, backslashes are handled in a non-obvious fashion inside of backticks. – calico_ Dec 14 '17 at 01:00
  • 1
    Your summation about a=\$b is wrong. Interpretation is not recursive, hence when you echo $a you get back the string which was assigned to a which happens to be '$b', so it was the same as writing a='$b'. You could go on to use eval, eval echo $(echo $a) – grail Dec 14 '17 at 02:07
  • 3
    Quite simply, a command substitution starts a new quoting context. The contents of the command substitution are processed independently of any quotes in which it appears. – chepner Dec 14 '17 at 03:05
  • calico_: Thanks for pointing it out, I mistakenly wrote "single" instead of "inner". – Larry Dec 14 '17 at 15:07
  • chepner: could you point me to some reference which describes quoting contexts or this behavior more in detail? As I've stated in my post, I have already encountered this term when searching for answers, alas without any explanation. It would be best if I knew how it fits into all the stuff described by the Bash manual page. – Larry Dec 14 '17 at 15:35
  • calico_: The page linked by you does not really contain anything surprising, nor anything that would answer the question. It also makes a statement that seems wrong to me, namely that nesting double quotes in backticks is not possible with Korn shells, even though I am using a Korn shell and it seems to work fine using the "dirname" example. Maybe the author meant "some" Korn shells, and not "all". However, in conjunction with "$()", it does contain some more explanation of what a quoting context is, albeit it does not mention that it applies to backticks as well. – Larry Dec 14 '17 at 16:48

1 Answers1

4

POSIX has this to say in 2.2.3 (emphasis mine):

` (backquote)

The backquote shall retain its special meaning introducing the other form of command substitution (see Command Substitution). The portion of the quoted string from the initial backquote and the characters up to the next backquote that is not preceded by a <backslash>, having escape characters removed, defines that command whose output replaces "`...`" when the word is expanded. Either of the following cases produces undefined results:

  • A single-quoted or double-quoted string that begins, but does not end, within the "`...`" sequence

  • A "`...`" sequence that begins, but does not end, within the same double-quoted string

This, to me, pretty much defines what other people might (informally?) call a quoting context comprising everything within two consecutive backquotes.

In a way, the backquote is the fourth quote in addition to single quote, double quote and backslash. Note that within double quotes, single quotes lose their quoting capability as well, so it should be no surprise that backquotes change the function of double quotes within them.

I tried your example with other shells, like the Almquist shell on FreeBSD and zsh. As expected, they output some path with.

Community
  • 1
  • 1
Jens
  • 69,818
  • 15
  • 125
  • 179