31

When I use the "tab" key in bash (when you have started to type the filename and you want it to complete), bash escapes the filename correctly, and if I use exactly that "escaped" filename, it works.

For Instance:

An-Beat - Mentally Insine (Original Mix).mp3 => After bash Escapes It Using "TAB" An-Beat\ -\ Mentally\ Insine\ \(Original\ Mix\).mp3

I'm search for a function for bash that will escape a filename the same way "tab" escapes filenames.

codeforester
  • 39,467
  • 16
  • 112
  • 140
M.A.G
  • 321
  • 1
  • 4
  • 7
  • 7
    http://mywiki.wooledge.org/XyProblem – Ignacio Vazquez-Abrams Apr 09 '11 at 21:47
  • 1
    Thanks @Ignacio, that sums of my feelings about this question far better than I could have said it. :) – sarnold Apr 09 '11 at 21:50
  • @Ignacio Indeed. This seems to be related to [this question](http://stackoverflow.com/questions/5607502/curl-complex-with-bash), and getting quotes in the variable value won't help at all -- he needs them *around* the variable substitution, and a way to loop over values, and... – Gordon Davisson Apr 09 '11 at 23:46

7 Answers7

43

Use printf (1):

x='a real \good %* load of c$rap'
x=$(printf '%q' "$x")
echo $x

will return 

a\ real\ \\good\ %\*\ load\ of\ c\$rap
sehe
  • 374,641
  • 47
  • 450
  • 633
  • If x is a parameter of a shell script, users can set x contains ('), and I must replace (') to (\'). Is there any easy way to handle such case? – zsxwing May 21 '12 at 08:13
  • @zsxwing it already does, just start with `x="'"` to demonstrate! (`printf '%q" "'"` prints `\'`) – sehe Oct 11 '13 at 07:16
  • 1
    %q in an unknown directive to printf on Ubuntu 15.04 – user3690202 Jun 27 '15 at 00:56
  • @user3690202 Use bash printf (it's a builtin, not the /usr/bin/printf or so if you have it) – sehe Jun 27 '15 at 01:29
  • A very special case that may not work : echo $(printf '%q' "test $2 3.jpeg")... If the file is originating from windows which allows $. $2 will be evaluated and return empty thus resulting in test\ \ 3.jpeg. – airboss Aug 15 '16 at 23:54
  • 2
    @airboss Huh. That's "Garbage In Garbage Out". The problem is not `printf` but you passing it invalid input. Just use `printf '%q' 'test $2 3.jpeg'` as you would anywhere where you want unexpanded string literals ([man bash](http://linux.die.net/man/1/bash)) – sehe Aug 16 '16 at 08:48
  • 2
    @airboss Also, if you just used the answer code as given (instead of changing it without understanding it): http://ideone.com/HdtYoJ – sehe Aug 16 '16 at 08:51
  • @sehe Sometimes the Garbage in cannot be controlled if its coming from users. This was just one of the example. Another made-up example of a filename: Name #1 [valid] : With.Special ä chars * 3 (2) incl & as ' well.jpg. And i see worse. – airboss Aug 16 '16 at 16:31
  • @airboss Your users don't edit the script. Here's all your inputs and more. http://ideone.com/RgXggz You can clearly see there's no problem with it, whatsoever. – sehe Aug 17 '16 at 21:01
  • @sehe, What does `%q` do? – Iulian Onofrei Jan 27 '17 at 11:22
  • @IulianOnofrei well not to state the obvious, but... it escapes the string value in a way that it's safe for use in a shell expression. See http://wiki.bash-hackers.org/commands/builtin/printf and of course [`man bash`](https://www.gnu.org/software/bash/manual/bash.html#Bash-Builtins) – sehe Jan 27 '17 at 11:33
  • Thanks, `Print the associated argument shell-quoted, reusable as input` is what I was looking for. – Iulian Onofrei Jan 27 '17 at 11:52
  • I've found that this works perfectly with one notable exception: paths that begin with tilde. Anyone have a workaround for that? – Compholio Nov 03 '21 at 19:58
  • @Compholio it works fine, unless you don't supply the input in a context that doesn't do tilde expansion: http://coliru.stacked-crooked.com/a/75bcbc0d07d060dc – sehe Nov 03 '21 at 20:30
  • @sehe With your example, I would expect this to work: `eval ls $myvar` but because of the escaped tilde it does not. – Compholio Nov 04 '21 at 17:48
  • @Compholio "It does not work" - how? http://coliru.stacked-crooked.com/a/2bdaee7d64b12463 Code talks. It does work, and besides, `eval` is it's own can of worms, so maybe separate the concerns of escaping and `eval` which introduces its own expansion context(s) – sehe Nov 04 '21 at 18:55
9

I'm going to elaborate on sehe's response on this one.

If you want to pass the argument to be converted as a shell script parameter, encase the parameter in "'s.

#!/bin/bash
x=$(printf '%q' "$1")
echo $x

I really like the printf solution, since it does every special character, just like bash.

bvarner
  • 363
  • 3
  • 9
4

The solution from "sehe" works fine, in addition, you can also use double quotes (") instead of single apostrophe (') to by able to use variables:

x="a real \good %* load of crap from ${USER}"
echo $(printf '%q' "$x")

Of course the string may not contain $ or " itself or you have to escape those manulally by splash \$.

panticz
  • 2,135
  • 25
  • 16
4
$ string="An-Beat - Mentally Insine (Original Mix).mp3"
$ echo ${string// /\\ }
An-Beat\ -\ Mentally\ Insine\ (Original\ Mix).mp3
$ string=${string// /\\ }
$ echo ${string//(/\\( }
An-Beat - Mentally Insine \( Original Mix).mp3
kurumi
  • 25,121
  • 5
  • 44
  • 52
3
ls  --quoting-style=escape /somedir

this will output the escaped filenames, and also work with unicode characters, printf method does not work with Chinese, it outputs something like $'\206\305...'

Shen Wei
  • 31
  • 3
2

I may be a little late to the party but what worked for me is:

ls  --quoting-style=shell-escape

This way it also escapes characters like ! or '.

dubsauce
  • 43
  • 5
0

I'm search for a function for bash that will escape a filename the same way "tab" escapes filenames.

Solution to get escaped full paths

I've created a portable function to get all the escaped paths to all items in the current directory, tested on macOS and Linux (requires GNU bash installed).

escpaths() {
    find "$PWD" -maxdepth 1 -print0 | xargs -0 -I {} bash -c 'printf "%q\n" "$0"' {} | sort
}

Here's a rigorous test case scenario:

# Generate test directory, containing test directories and files.
mkdir 'test dir'; cd 'test dir'

mkdir '\dir  with\ backslashes\\\'
touch '\file with \\backslashes\'
touch 'An-Beat - Mentally Insine (Original Mix).mp3'
mkdir 'crazy(*@$):{}[]dir:'
mkdir 'dir with 字 chinese 鳥鸟 characters'
touch $'file\twith\tmany\ttabs.txt'
touch 'file @*&$()!#[]:.txt'
touch 'file
with
newlines.txt'

Executing escpaths in the test directory test dir gives the escaped output:

$'/.../test dir/file\nwith\nnewlines.txt'
$'/.../test dir/file\twith\tmany\ttabs.txt'
/.../test\ dir
/.../test\ dir/An-Beat\ -\ Mentally\ Insine\ \(Original\ Mix\).mp3
/.../test\ dir/\\dir\ \ with\\\ backslashes\\\\\\
/.../test\ dir/\\file\ with\ \\\\backslashes\\
/.../test\ dir/crazy\(\*@\$\):\{\}\[\]dir:
/.../test\ dir/dir\ with\ 字\ chinese\ 鳥鸟\ characters
/.../test\ dir/file\ @\*\&\$\(\)\!#\[\]:.txt

Solution to get escaped basenames only

This (also portable) function will get you the escaped basenames of all items in the current directory (this time excluding the current directory).

escbasenames() {
    find . -mindepth 1 -maxdepth 1 -exec printf '%s\0' "$(basename {})" \; | xargs -0 -I {} bash -c 'printf "%q\n" "${0#./}"' {} | sort
}

Running escbasenames in the same test directory test dir gives the escaped basenames:

$'file\nwith\nnewlines.txt'
$'file\twith\tmany\ttabs.txt'
An-Beat\ -\ Mentally\ Insine\ \(Original\ Mix\).mp3
\\dir\ \ with\\\ backslashes\\\\\\
\\file\ with\ \\\\backslashes\\
crazy\(\*@\$\):\{\}\[\]dir:
dir\ with\ 字\ chinese\ 鳥鸟\ characters
file\ @\*\&\$\(\)\!#\[\]:.txt

Final notes

Note that if the path/filename contains newlines or tabs, it will be encoded as an ANSI-C string. Autocompletion in the terminal also completes with ANSI-C strings. Example ANSI-C string autocompletion outputs would look like my$'\n'newline$'\n'dir/ or my$'\t'tabbed$'\t'file.txt.

Aeronautix
  • 301
  • 2
  • 13