Curl upload with spaces in filenames

Question

I am trying to use cURL to upload files with spaces in their filenames onto a dedicated server. I am using bash. In a previous project, I just gave up, removing all spaces from filenames. This is not feasible for this project.

Running cURL in verbose mode suggests that it stops when it reads my local file path:

curl -X PUT -u $USER:$PASS --data-binary @"$LOCAL_FILE" "$SERVER/remote.php/dav/files/$USER/$REMOTE_DIR/$REMOTE_FILE"

where $LOCAL_FILE is a path to a file on my local machine (with spaces), and $REMOTE_FILE also has spaces.

This gives:

Warning: Couldn't read data from file "/Users/my_account/somepath/with

Warning: spaces

which implies the command is taking "/Users/my_account/somepath/with" and "spaces" as two separate paths.

How can I solve this?

My full code:

#!/bin/bash -ex
# Local dir - note space in path
IMAGES_DIR="/Users/my_account/somepath/with spaces"

# Remote server, and credentials
SERVER="http://myserver"
REMOTE_DIR="mydir"
USER="myname"
PASS="mypass"

FILE="$1"

LOCAL_FILE="$IMAGES_DIR/$FILE"
REMOTE_FILE=urlencode $FILE

# Move file to the server
curl -v -X PUT -u $USER:$PASS --data-binary @"$LOCAL_FILE" "$SERVER/remote.php/dav/files/$USER/$REMOTE_DIR/$REMOTE_FILE"

# Check that file has made it
echo 'Waiting for file to be on server'

until [ $result > 0 ]
do
    result=$(curl -I "$SERVER/remote.php/dav/files/$USER/$REMOTE_DIR/$REMOTE_FILE" -u $USER:$PASS 2>/dev/null | grep Content-Length | awk -F ': ' '{print $2}')
    echo "."
    sleep 2
done

echo "File $FILE is now on server."

urlencode() {
    # urlencode <string>
    old_lc_collate=$LC_COLLATE
    LC_COLLATE=C

    local length="${#1}"
    for (( i = 0; i < length; i++ )); do
        local c="${1:i:1}"
        case $c in
        [a-zA-Z0-9.~_-]) printf "$c" ;;
        *) printf '%%%02X' "'$c" ;;
    esac
    done

LC_COLLATE=$old_lc_collate
}

I can't reproduce with my curl 7.29.0, which tested with `echo "some content">"file with spaces";nc -l -p 4242 -c 'echo -e "HTTP/1.1 200 OK\n\nresponse"' & curl -v http://localhost:4242/ --data-binary @"file with spaces"` ; you might want to check what's wrong with `curl -v` at first (which I don't expect will reveal much), then with `bash -x` to inspect the actual `curl` command — Aaron, Aug 09 '17 at 13:30

score 2 · Answer 1 · edited Dec 21 '21 at 21:43

In the Windows version of curl 7.55.0 (should be in higher versions also) the below command will work:

local Windows path: path enclosed in double or single quotes

server URL: path enclosed in double or single quotes and each space replaced with '%20' an example to upload tes st.txt is given below

curl -username:password -T "./Test/te st.txt" "http://server-url/folder1/folder2/te%20st.txt"

score 0 · Answer 2 · answered Aug 10 '17 at 07:12

Did you tried changing the value of IFS in your script? Normally its value is set to $' \t\n', which includes space. Set it to nothing like so: IFS= and no field splitting is performed at all. You can get the default value back by executing unset IFS.

Example:

var="a   b   c   d"
echo $var
a b c d
IFS=
echo $var
a   b   c   d

score 0 · Answer 3 · answered Nov 04 '19 at 19:05

TL;DR : the problem still occurs on windows with filenames containing illegal characters, like | > < :

As @Jerin points out, curl on windows (at least in versions > 7.55.0) will accept spaces in filenames.

However, with curl 7.57.0 (x86_64-w64-mingw32), I'm observing that

curl -X POST http://localhost:8080/ -H "Content-Type: application/json" --data-binary "@my_dir/2019-11-04>.json"

gives me

Warning: Couldn't read data from file "my_dir/2019-11-04>.json",
Warning: this makes an empty POST.

And doesn't pass the file as POST data.

Curl upload with spaces in filenames

3 Answers3