0

I'm trying to read the output of a RESTful API status on the progress of a download and print that out. As soon as the download is finished the status changes from "running" to "terminated regularly" and the "progress" indicator is not shown anymore. Therefore the loop end with an empty variable $progress.

So the following script is correctly showing the progress of the download, but as soon as the download is finished the script ends abruptly and doesn't show the "Download finished."

Assume a download is running on session 1.

while true; do
    status=$(curl -s -u admin:admin -H "Accept:application/json" -X GET \
    -i http://1.2.3.4/sessionInformation/1/status | grep "Status" | sed 's/    "Status"://g' | sed 's/"//g')

        while [[ $status != "terminated regularly"  ]]; do
            progress=$(curl -s -u admin:admin -H "Accept:application/json" -X GET \
            -i http://1.2.3.4/sessionInformation/1/status | grep "Progress" | sed 's/["Progress":"      %]//g')
            echo -ne "Download ${progress} % finished.\r"
        done
        echo "Download finished."
    break
done

Here's the output of the script running with "set -x":

+ progress=99
+ echo -ne 'Download 99 % finished.\r'
+ [[ running, != \t\e\r\m\i\n\a\t\e\d\ \r\e\g\u\l\a\r\l\y ]]
++ curl -s -u admin:admin -H Accept:application/json -X GET -i http://1.2.3.4/sessionInformation/1/status
++ grep Progress
++ sed 's/["Progress":"      %]//g'
+ progress=

I'm running MacOS Ventura 13.4.1 and Bash 3.2.57(1).

I tried wrapping the whole thing in various if-statements and/or while loops, but no mater what else I read as an indicator of the download being finished and the status having changed, the problem is always the empty $progress variable. I also tried checking $progress using -z, but that didn't work. I just want the loop to dislay the progress and as soon as it's done downloading to proceed with whatever, at least printing another prompt, that it has finished.

dataprolet
  • 1
  • 2
  • Looks to me like it's getting stuck in the call to `curl` at the `progress=$(curl -s ...` line - [add a timeout](https://unix.stackexchange.com/a/94612/133219) to see if that helps. Btw your code has an infinite loop with `while [[ $status != "terminated regularly" ]]; do` since `status` never changes inside the loop and you don't `break` out of it. – Ed Morton Aug 22 '23 at 12:57
  • status is not modified in inner while loop, so should loop forever, maybe the inner while could be changed by a if and a continue should be added before fi (instead of done) – Nahuel Fouilleul Aug 22 '23 at 13:02
  • also grep + sed may be changed with `grep -Po '"Status":\s*"\K[^"]*'` and `grep -Po '"Progress":\s*"\K[^"%]*'` – Nahuel Fouilleul Aug 22 '23 at 13:09
  • In general, using syntax-unaware tools to parse data out of strings in a well-defined format (be it JSON, or YAML, or anything else) is a serious source of headaches, both for you and for whoever's on the other side of the network connection. (I've had a customer who used this kind of hand-rolled "parser" in a past life, and they were a source of nothing but headaches; if you're getting a formatted data structure, use a real parser -- for JSON, `jq`, for XML, `xmlstarlet`, for YAML, `yq`, etc -- to interpret it). – Charles Duffy Aug 22 '23 at 13:13
  • 2
    @NahuelFouilleul FYI `grep -Po` is non-portable (requires GNU grep for both `-P` and `-o`) whereas `sed -n 's/.*"Status":"\([^"]*\).*/\1/p'` would work using any sed on every Unix machine. Any time you want to combine grep+sed commands, using sed only or switching to awk are usually better options than using grep only. – Ed Morton Aug 22 '23 at 13:29
  • @EdMorton, i agree, it depends on the context, that's the quickiest, otherwise i would go for perl, or jq. But i think the issue here is just logic with the two nested whiles, as in my first comment – Nahuel Fouilleul Aug 22 '23 at 13:41
  • Either way, since nobody but the OP knows what curl is returning, we can't suggest changes to the `grep`/`sed`/&c and test them ourselves. (If the server really _is_ returning JSON as the headers request, then using `jq` instead of `grep` would be the Right Thing, but to provide a tested jq expression there too there are details the question doesn't provide). – Charles Duffy Aug 22 '23 at 14:35
  • Thanks everybody. I guess one problem is that the inner loop is infinite since $status is not updated. I tried to make the inner loop an if-statement, but that doesn't work either. Also it's necessary to keep this script as portable as it gets, because it's supposed to be run on changing machines, on which I don't want to install third-party software. So unfortunately I have to work with grep and sed. Here's an example output from after the download has finished: https://pastebin.com/raw/W8Q1bdmX. – dataprolet Aug 28 '23 at 08:33

0 Answers0