1

I'm making a wrapper script for a command in bash and I want to pass string arguments with whitespace while keeping it as a string inside the bash file.

I've seen many questions which refer to passing a string as one item to a bash script, but not how to maintain the string should the intention be for it to remain as a string when being usedin the script.

I've also tried numerous different ways of quoting and escaping characters to see if it makes any difference

An example of how my script is laid out is

#!/usr/bin/env bash
exec my_program /path/to/code.py "${@}"

When I execute my_script -arg "A string", the desired behaviour is to have the wrapped command to execute as my_program /path/to/code.py -arg "A string" but instead it runs as my_program /path/to/code.py -arg A string resulting in error: unrecognized arguments: string.

Is there some way that I can ensure that, when passed a string, it will maintain the string all the way down?

A Minimal, Complete, and Verifiable example:

bash_strings.sh

echo "${@}"

Output

$ bash bash_strings.sh --test "A String"
--test A String
Ray Barrett
  • 73
  • 3
  • 10
  • 2
    If you have `"A string"` parsed by bash, it keeps `A string` as a single array element, and so it correctly becomes a single element in `sys.argv` in Python. That's exactly what is *supposed* to happen. The quotes you typed are bash syntax telling bash to keep the items together, not part of the string itself. – Charles Duffy Apr 09 '19 at 16:20
  • 2
    The premise of your question is faulty. `"$@"` is exactly what you need to preserve `"A string"` as a single word. If it's being split into two words it's happening somewhere else and we need to see a [mcve] that demonstrates the problem. The script you posted is fine as is. – John Kugelman Apr 09 '19 at 16:22
  • ...it's like if you used `subprocess.Popen(['someprogram', '''foo bar'''])` in Python -- you don't expect the triple-quotes around `foo bar` to be part of the command that's run, right? Same thing in bash; syntactic quotes (as opposed to literal ones) are consumed at parse-time, not passed to the invoked program. – Charles Duffy Apr 09 '19 at 16:24
  • I think I see what you're referring to. Are you saying that I should not expect to see the quotes in the output because it's similar to how python would do `print("A String)`? If so, I see your point. In the case of Popen, I would likely pass a repr of the string, or `'"foo bar"'` so that I get the desired results. In this case I don't know how I would achieve this in bash. – Ray Barrett Apr 09 '19 at 16:56
  • If some use of `Popen` is your goal, add the example to your question. You don't want to construct a `bash` shell line if you can at all avoid it; e.g. `a = "A String"; p = Popen(["myScript", "-arg", a])` will do what you want. The value of `a` is never processed as part of a shell command line; it is passed directly to `execve`, and `myScript` will receive the correct argument. – chepner Apr 09 '19 at 17:08
  • @RayBarrett so to clarify, you want the output in your example to be `--test "A String"` ? – Nick Vitha Apr 09 '19 at 17:11
  • @RayBarrett, do you want *bash's* `repr()` of the string, or a JSON representation, or a Python representation? We can do all those things, but they're three different things. – Charles Duffy Apr 09 '19 at 17:17
  • ...to be clear, `echo "$@"` is itself inherently buggy; it hides the differences between `echo "hello" "world"` and `echo "hello world"`, and should never be used for anything important. – Charles Duffy Apr 09 '19 at 17:17
  • @RayBarrett, ...see also [BashPitfalls #14](http://mywiki.wooledge.org/BashPitfalls#echo_.24foo) (mostly about use of `echo` with inadequate quotes, but not entirely irrelevant here either), and the APPLICATION USAGE section of [the POSIX spec for `echo`](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html), which explicitly advises using `printf` instead. – Charles Duffy Apr 09 '19 at 17:22
  • 1
    `(( $# )) && printf ' - %s\n' "$@"` is another easy way to get a list of your command-line arguments that doesn't confuse where they start and end. – Charles Duffy Apr 09 '19 at 17:25
  • @chepner: I'm not looking to use Popen here, just responding to the Popen example provided by Charles Duffy – Ray Barrett Apr 10 '19 at 08:55
  • @NickVitha: Yes, I'm looking for the output to retain the quotes on the string. – Ray Barrett Apr 10 '19 at 08:57
  • @RayBarrett, ...so, let's say that instead of `bash_strings` being written in bash, it were written in Python. I assume you'd implement it as something like `print(' '.join(repr(s) for s in sys.argv))`, relying on the `repr()` call to *add* quotes to make a human-readable representation of the data, not `print(' '.join(*sys.argv))`, throwing away boundaries and thus resulting in an unclear result. – Charles Duffy Apr 10 '19 at 17:14
  • @RayBarrett, ...if someone came to you asking how to call their Python program in such a way as to make `print(' '.join(*sys.argv))` emit a result with quoting and escaping (instead of fixing the Python program to use `repr()`), you'd want to know *why* they're asking for something so odd, rather than accepting the data in its actual form and worrying about the representation at print-time, right? That's where I'm at right now. – Charles Duffy Apr 10 '19 at 17:17
  • @CharlesDuffy I'm trying to execute this is an environment which is being dynamically resolved, so the parameters I'm providing are being passed through several layers. My goal is to maintain the string when it's being passed through N layers of script. (I have no control over the environment in question). Potentially a PEBKAC issue, I'll need to figure out more about how the system is resolving environments and passing arguments. – Ray Barrett Apr 11 '19 at 09:26
  • @RayBarrett, ...so, here's the thing. Sometimes you *do* have a real reason to pass code between processes as code, but you basically never want to have it spread across multiple arguments when doing so. If in your *outer* process generating the code you run `printf -v cmd_str '%q ' "$@"` on the list, and pass `"$cmd_str"` as *exactly one* argument, then you get something that the inside process can pass to `eval` exactly-as-is (quoting it: `eval "$1"`, not `eval $1`). Doing anything that would split `eval`-able code across an argument list is pretty much inherently the Wrong Thing. – Charles Duffy Apr 11 '19 at 12:58
  • @RayBarrett, ...in our Python analogies, you're asking to get the right (quoted) output from `print(sys.argv[1])`, which is a lot more reasonable than asking to get it from `print(' '.join(*sys.argv))`. – Charles Duffy Apr 11 '19 at 12:59
  • I would also strongly suggest considering whether the right way to pass code through your process tree is in an exported function. `myfunc() { something --test "A string"; }; export -f myfunc` in one process, and then any child process that (1) is bash, and (2) inherits the unfiltered environment variable set will be able to run `myfunc`. (If you try to use that pattern and it doesn't work, see https://stackoverflow.com/questions/38079864/regression-exported-bash-function-lost-after-going-through-another-process, discussing how condition-2 can sometimes fail to come about). – Charles Duffy Apr 11 '19 at 13:02
  • (Ugh, meant to say "pass code between processes as text" several comments up; *grumble* edit window *grumble*). – Charles Duffy Apr 11 '19 at 13:04
  • ...if you *are* trying to pass code in a single argument already, anything that splits your `$1` out into something spread over `"$@"` is going to have other side effects, some of them nasty -- so if your environment is doing that to you, that environment is *actively buggy* and we really want to find a way to, if not fix it, work around those bugs conclusively. Unfortunately, that's hard to do without enough details to reproduce the issue, or at least enough details to identify workarounds. (*Are* environment variables passed through? If so, they might survive intact even if args don't). – Charles Duffy Apr 11 '19 at 13:05
  • I've discovered today that the arguments I'm supplying to the code are going `console->bash->bash->python->bash->python`, so I'm thinking the environment is actively buggy and I'll likely be looking to simplify that as a means of "fixing" this issue or, at least, making it easier to reason about. @CharlesDuffy your function idea is an interesting one, I'll have a look into that pattern once the environment is cleaned up. Thanks for your suggestions everyone! – Ray Barrett Apr 11 '19 at 16:32

3 Answers3

3

Your bash_strings.sh is buggy, insofar as echo "$@" throws away data (making ./yourscript "hello world" look identical to ./yourscript "hello" "world"). To accurately reflect a command's argument list, use printf %q to generate a quoted version, as follows:

#!/usr/bin/env bash
(( $# )) || exit
printf '%q ' "$@"; echo

...will, for ./bash_strings --test "A string", emit as output:

--test 'A string'

...or, potentially, a different semantically-identical representation such as:

--test A\ string

...either way, reflecting that the original quoting was genuinely passed through.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • 1
    This problem is inherent to `echo`. For example, note that `echo --test "A String"`, `echo --test A String`, and `echo "--test A String"` all print exactly the same thing. (But using `printf '%q '` with those same argument lists will make the difference clear.) – Gordon Davisson Apr 09 '19 at 17:31
  • Edited to make that more clear; thank you. (As this answer is flagged community-wiki, do also feel free to jump in and edit it yourself). – Charles Duffy Apr 09 '19 at 17:35
-2

Bash will take the quotes out. You have to insert them again. But, if you escape the quotes in wrapper, every argument will be interpreted inside quotes:

exec my_program /path/to/code.py \"${@}\"

will become

my_program /path/to/code.py "-arg A string"

which you also don't want.

I think the best approach is, if your wrapper works to a specific program, do a case with the different flags, like: 

while getopts "arg:r:v" arg; do
  case "$arg" in
    arg)
    exec my_program /path/to/code.py -arg $OPTARG
    ;;
    r)
    exec my_program /path/to/code.py -r $OPTARG 
    ;;
    v)
    exec my_program /path/to/code.py -v $OPTARG 
    ;;
  esac
done
samthegolden
  • 1,366
  • 1
  • 10
  • 26
-2

Edited as OP has edited the question for clarity.

If you change your input to:

bash bash_strings.sh --test "\"A String\""

It will work, for this specific example. Bash is stripping quotes somewhere in there.

Nick Vitha
  • 466
  • 2
  • 9
  • When you run `"A STRING"` in bash, the quotes are **part of the bash syntax**, not part of the command. It's completely correct for them not to be in `sys.argv`. – Charles Duffy Apr 09 '19 at 16:21
  • @CharlesDuffy I'm not quite sure what you're saying. – Nick Vitha Apr 09 '19 at 16:25
  • I'm saying that the OP's original code already had working syntactic quotes causing the argument list to be passed verbatim, and that adding *literal* quotes is doing harm, not good. – Charles Duffy Apr 09 '19 at 16:27
  • ...indeed, `"\"$@\""` prefixes a `"` before the first argument and a `"` after the last argument, but it leaves the others unmodified, so `./yourprogram "foo bar" "baz qux" "third arg"` becomes `['"foo bar', 'baz qux', 'third arg"']`, with a `"` added before the `foo` and another added after the final `arg`. I'm hard-pressed to see how that's a useful or valuable result. – Charles Duffy Apr 09 '19 at 16:28
  • It might be useful if his `my_program` screws with the arguments in some way. – Nick Vitha Apr 09 '19 at 16:29
  • ... but if so then his program is not a complete example as someone said above. – Nick Vitha Apr 09 '19 at 16:30
  • Yup. There are good reasons that [How to Answer](https://stackoverflow.com/help/how-to-answer) advises one to "answer well-asked questions". Making assumptions that the OP must be doing something weird enough to make their question useful even when no such thing is clearly described leads to giving bad advice to anyone *not* doing that hypothetical very-weird thing. – Charles Duffy Apr 09 '19 at 16:32
  • @CharlesDuffy I'll leave my answer for now and edit it if the asker responds. If not, I'll probably delete it or modify it. – Nick Vitha Apr 09 '19 at 16:36
  • If they do clarify enough to allow a good answer, @-notify me after those edits are done and I'll adjust my voting appropriately. :) – Charles Duffy Apr 09 '19 at 16:37
  • @CharlesDuffy Well, I don't thnk you should change your vote, but OP has put an example in that shows the issue he's having. – Nick Vitha Apr 09 '19 at 17:16