cURL: How to display progress information while uploading?

Question

I use the following syntax to upload files:

curl --form upload=@localfilename --form press=OK [URL]

How to display the progress? Thx.

http://curl.haxx.se/docs/manpage.html See the section on progress meter. I suspect that you need to pipe the output to a file, though am not positive. — Corbin, Apr 02 '12 at 08:27

score 89 · Accepted Answer · edited Dec 04 '19 at 19:16

This is what I use in one of my build scripts:

curl "${UPLOAD_URL}" \
    --progress-bar \
    --verbose \
    -F build="${BUILD}" \
    -F version="${VERSION}" \
    -F ipa="@${IPA};type=application/octet-stream" \
    -F assets="@-;type=text/xml" \
    -F replace="${REPLACE}" \
    -A "${CURL_FAKE_USER_AGENT}" \
    <<< "${ASSETS}" \
    | tee -a "${LOG_FILE}" ; test ${PIPESTATUS[0]} -eq 0

The -F and -A options will probably not be of interest to you, but the helpful parts are:

curl "${UPLOAD_URL}" --progress-bar

which tells curl to show a progress bar (instead of the default 'progress meter') during the upload, and:

 | tee -a "${LOG_FILE}" ; test ${PIPESTATUS[0]} -eq 0

which appends the output of the command to a log file and also echo's it to stdout. The test ${PIPESTATUS[0]} -eq 0 part makes it so that the exit status of this line (which is in a bash script) is the same exit code that the curl command returned and not the exit status of the tee command (necessary because tee is actually the last command being executed in this line, not curl).

From man curl:

PROGRESS METER
       curl normally displays a progress meter during operations, indicating the
       amount of transferred data, transfer speeds and estimated time left, etc.

       curl  displays  this  data to the terminal by default, so if you invoke curl
       to do an operation and it is about to write data to the terminal, it disables
       the progress meter as otherwise it would mess up the output mixing progress
       meter and response data.

       If you want a progress meter for HTTP POST or PUT requests, you need to
       redirect the response output to a file, using shell redirect (>), -o [file]
       or similar.

       It is not the same case for FTP upload as that operation does not spit out
       any response data to the terminal.

       If you prefer a progress "bar" instead of the regular meter, -# is your
       friend.

OPTIONS
       -#, --progress-bar
              Make curl display progress as a simple progress bar instead of the
              standard, more informational, meter.

NB! `tee` command required to get info about download progress. if you don't want use `tee` then just use `grep -v '^uniqueStringNeverHappens$'` command. more info: http://stackoverflow.com/a/17178410/751932 — Maxim Kholyavkin, Sep 16 '14 at 07:44
This does not seem to work for me :( EDIT: nevermind, the problem was that I wanted to time how long it takes, so I used `time curl --progress-bar --verbose ...` which prevents the progress bar from being shown. Running it without time works. — Grey Panther, Jul 26 '16 at 03:56
"If you want a progress meter for HTTP POST or PUT requests, you need to redirect the response output to a file, using shell redirect (>), -o [file] or similar." This should be highlighted, gets me everytime. — jmoz, Feb 02 '19 at 03:48
`--progress-bar` worked for me, but remember o put your URL in quotes — programmingaddict, Feb 21 '23 at 13:04

score 83 · Answer 2 · edited Apr 13 '23 at 06:53

83

All other answer here have the problem that they require you to write the original output of curl into a (log) file. This may however not be wanted in all cases.

The issue is that curl hides the progress bar/meter, when a server response is expected, which is then written into stdout. So basically you can redirect the output to a file to show the bar again. However we do not want that, so we can do this:

curl --progress-bar -T "${SOME_LARGE_FILE}" "${UPLOAD_URL}" | cat

Curl's output is passed to cat which writes the output to the console (which we want to see the progress bar and also the server response).

Note that the curl devs of course did not hide the progress bar for fun. In this case here you may not always see the server result or it may only be shown a few seconds (as the progress bar is shown again afterwards), but if you do not care about this, the solution is a nice one.

edited Apr 13 '23 at 06:53

Juuso Ohtonen

8,826
9
65
98

answered Jan 25 '17 at 19:31

rugk

4,755
2
28
55

6

I think they just hid it for fun, because you are explicitly asking for the progress bar with `--progress-bar`. – Chloe Mar 27 '18 at 01:16
8

Why `curl --progress-bar ... | tee /dev/null`? In my case `curl --progress-bar ... | cat` seems to be working just as well. – user000001 Nov 22 '19 at 09:53
1

@user000001 Of course, that's even easier. Thanks. :) – rugk Nov 25 '19 at 12:59
You can also simply use "tee" no need for the /dev/null – Trent Lloyd Sep 15 '20 at 05:32
1

@user672009 your edit changed the code example but not the explanation of the answer, and now it's unclear. – Chirlo Mar 13 '23 at 08:56
`type` instead of `cat` for windows – Mattwmaster58 Jun 06 '23 at 16:24

score 53 · Answer 3 · answered Sep 03 '16 at 01:22

53

I had trouble with the accepted answer's command redirection and found the -o option will place the response output in a file which allows the progress bar to show up.

curl --progress-bar \ 
     -o upload.txt \
     -H ${SOME_AUTH_HEADER} \
     -T ${SOME_LARGE_FILE} \
     "${UPLOAD_URL}"

Just another option to get the desired result.

NOTE: Emphasis on this line from the man page is important to understand the root cause of why the progress bar is not showing when just specifying --progress-bar.

      If you want a progress meter for HTTP POST or PUT requests, 
      you need to redirect the response output to a file, 
      using shell redirect (>), -o [file] or similar.

answered Sep 03 '16 at 01:22

Martin Woolstenhulme

3,968
4
24
25

2

Thank you! Omg took way too long to figure this out. – Melroy van den Berg Oct 02 '20 at 19:57
2

This should be the accepted answer. – Onnonymous Oct 15 '21 at 15:31
Thanks very much! I have been scratching my head in where the progress bar has gone? Yes, redirect the output!! Thanks again! – HAltos Dec 03 '21 at 01:15
Also, if one only wants to see the progress bar, one can do this: $ curl --progress-bar .... > /dev/null – HAltos Dec 03 '21 at 01:42
What about progress bar separately from stderr stream where all verbosity goes on? Or how to move progress bar from stderr into stdout? – Andry May 07 '22 at 13:12

score 3 · Answer 4 · answered Jul 30 '21 at 18:04

3

To show progress bar on Windows: curl --progress-bar --upload-file Myfile.zip "https://my.upload.site/Some/Upload/Path/Myfile.zip" | type

answered Jul 30 '21 at 18:04

LukeInDC

31
2

score 1 · Answer 5 · answered May 07 '22 at 14:37

For whose who wants to move progress from stderr to stdout:

curl ... -o ... ... 2>&1 | sed -E 's/\r([^\n])/\n\1/g' | grep -P '^(?:  [% ] |(?:  |  \d|\d\d)\d |[<>] )'

tested on download

The sed has to be used to ignore blank lines by replacing CR by LF. This is required for uniform parse the curl output in both verbose or non verbose mode.

cURL: How to display progress information while uploading?

5 Answers5

Linked