Pass wildcard to scp in a wrapper bash script

Question

I am writing a bash wrapper for scp'ing into and from a certain host with a certain username, like:

johny@bonjour:~/bin$ cat scpphcl 
#!/bin/bash

download=false
upload=false
local=""
remote=""

usage()
{
    echo "Usage: $0 -d[-u] -l <LocalPath> -r <RemotePath>"
    exit 1
}

while getopts "h?dul:r:" opt; do
    case "$opt" in
    h|\?)
        usage
        ;;
    d)
        download=true
        upload=false
        ;;
    u)
        download=false
        upload=true
        ;;
    l)
        local=$OPTARG
        ;;
    r)
        remote=$OPTARG
        ;;
    esac
done

if [[ -z $local || -z $remote ]]; then
    echo "Need to provide local and remote path."
    usage
fi

if $download; then
    scp somebody@somehost:"$remote" $local
elif $upload; then
    scp $local somebody@somehost:"$remote"
else
    echo "Neither download nor upload?"
    exit 1
fi

if [[ $? -ne 0 ]]; then
    echo "Something wrong happened in the scp process."
    exit 1
fi

exit 0

It works well with the usual filenames, but if there is any wildcard in the local filename field, it will not work right.

johny@bonjour:~/test$ scpphcl -u -l * -r /u/somebody/temp
Need to provide local and remote path.
Usage: /Users/johny/bin/scpphcl -d[-u] -l <LocalPath> -r <RemotePath>

There is a walkaround, using sinqle quotes around the local file argument if there is a wildcard in it:

johny@bonjour:~/test$ scpphcl -u -l '*' -r /u/somebody/temp

But even this walkaround will not work, if the command is issued outside the folder test:

johny@bonjour:~/test$ cd ..
johny@bonjour:~$ scpphcl -u -l 'test/*' -r /u/somebody/temp

This doesn't work and will hang in the scp process.

Any help in how to pass the wildcard in local filenames with the bash wrapper?

You might need to escape the wild-card to let it be expanded remotely and _not_ locally — Inian, Feb 27 '18 at 15:25
@Inian Neither `scpphcl -u -l 'test/\*' -r /u/somebody/temp` nor `scpphcl -u -l test/\* -r /u/somebody/temp` works. — Qiang Xu, Feb 27 '18 at 15:43
You'll need to quote `"$OPTARG"` (thus), too. And `"$local"`, and `"$remote"`. Just quote properly, everywhere. Actually, you want `local=($OPTARG)`, and then use `"${local[@]}"` in the command. But call it something else - `local` is a Bash keyword. — Toby Speight, Feb 27 '18 at 15:53
@TobySpeight Thank you very much, Toby. It works. Also, thanks for your suggestion of giving the local path a name other than `local`. I didn't know it is a bash keyword. Could you put your comment into the answer section so that I can mark it as the solution? It would be better if you add some explanation. I guess `localfile=($OPTARG)` probably treat the wildcard expansion as an array. Later, when it was used in the `scp` command, is `${localfile[@]}` the way to dereference the array? — Qiang Xu, Feb 27 '18 at 16:26
Yes, that's an array. The way to expand the array is `"${localfile[@]}"` (those double-quotes are important!). — Toby Speight, Feb 27 '18 at 16:39

Toby Speight · Accepted Answer · 2018-02-27T17:00:18.243

2

It's probably best not to require your users to quote wildcard patterns. I'd instead change the interface of your program to accept any number of local paths, after the option arguments:

echo "Usage: $0 [-d|-u] [-r <RemotePath>] <LocalPath>..."

When reading options, consume them with shift:

while getopts "h?dur:" opt; do
    case "$opt" in
    h|\?)
        usage
        exit 0
        ;;
    d)
        download=true
        upload=false
        ;;
    u)
        download=false
        upload=true
        ;;
    r)
        remote="$OPTARG"
        ;;
    *)
        usage >&2
        exit 1
        ;;
    esac
done
shift $((OPTIND-1))

Now the remaining positional arguments are the local filenames (and can be accessed with "$@" - note the all-important double-quotes there):

if test -z "$*"  # no LocalPath arguments!
then usage >&2; exit 1
elif $download
then exec scp somebody@somehost:"$remote" "$@"
elif $upload
then exec scp "$@" somebody@somehost:"$remote"
fi

edited Feb 27 '18 at 17:00

answered Feb 27 '18 at 16:38

Toby Speight

27,591
48
66
103

Perfect solution! More concise and more user-friendly! Thanks a bunch, Toby! – Qiang Xu Feb 27 '18 at 16:58
Another question: what's the gain of using `exec` to do the `scp`, instead of calling it directly as I did before? – Qiang Xu Feb 27 '18 at 17:11
1

It saves a fork - but does mean that you don't get to do anything afterwards (so your helpful "something went wrong" message doesn't get shown - I'm hardy enough that `scp`'s own error message is enough, but your users might differ there). Another advantage is that when the exit status matters, your users get that status directly (rather than just `0` or `1` as in your version - you'd have to do some work to preserve the status code). – Toby Speight Feb 27 '18 at 17:17

Pass wildcard to scp in a wrapper bash script

1 Answers1