bash how to error out when subshell error

Question

I have a script with the following command to upload a bunch of zip files to a site:

find . -name "*.zip" -exec echo {} \; -exec sh -c 'err=$(curl -s --data-binary "@{}" http://mystorage.com | jq -r ".error"); if [ -z $err ] || [ $err = "file already exists" ]; then exit 0; else exit 1; fi' \;

The intention is that if any file fail to upload with the reason other than "file already exists" then the script must fail. However, if i run this command alone, it never exit with 1. My guess is that the subshell opened in the 2nd -exec returns 1 but the -exec ignore the return status and return 0 for the whole find command. Is there a way to make my command fail when the subshell fail?

chepner · Accepted Answer · 2022-06-23T19:38:46.313

1

I wouldn't bother with find for this. Just use an ordinary loop (with the globstar option to search recursively, if necessary).

shopt -s globstar nullglob

for f in **/*.zip; do
  err=$(curl -s --data-binary "@$f" http://mystorage.com | jq -r ".error")
  if [ -n "$err" ] && [ "$err" = "file already exists" ]; then
    exit 1
  fi
done

Note that you don't want to exit 0 when the first job succeeds; just do nothing and let the next file be uploaded.

edited Jun 23 '22 at 19:38

answered Jun 23 '22 at 18:29

chepner

497,756
71
530
681

thanks, it makes send to rewrite using a loop – lnthai2002 Jun 23 '22 at 19:23

score 0 · Answer 2 · answered Jun 23 '22 at 18:50

You need to look at the four exec forms find can use: https://www.man7.org/linux/man-pages/man1/find.1.html

-exec command {} \;
-exec command {} \+
-execdir command {} \;
-execdir command {} \+

They all have different behaviors regarding their boolean value in the search boolean expression.

My brief interpretation (I have only read the manpages, I have not actually tried this):

Commands that end with a semicolon result in the -exec term having a true or false value, and do not effect the exit value of find even when the wrapped command has a nonzero exit value.
Commands that end with a plus cause find to exit with a nonzero value when a wrapped command has a nonzero result value.

So I think you want to switch to the -exec form where the command ends with a '+'.

But really I think you need a better multithreaded script that can handle each failure separately, and remember which ones it has successfully uploaded.

Thanks, i also tried the '+' instead of '\;' but it does not work with the subshell since it requires the {} to be in the exec command itself. I think it is best just to rewrite with a loop — lnthai2002, Jun 23 '22 at 19:22

score 0 · Answer 3 · answered Jun 23 '22 at 19:25

Using GNU find:

find . -name '*.zip' -print -exec sh -c '
err=$(curl -s --data-binary "@$1" http://mystorage.com | jq -r ".error")
[ -z "$err" ] || [ "$err" = 'file already exists' ] || exit 1
' _ {} \; \
-o -quit

Note that this exits if .error is empty. That may be what you want, but maybe you want this (or similar):

find . -name '*.zip' -print -exec sh -c '
    response=$(curl -s --data-binary "@$1" http://mystorage.com) || exit 1
    [ "$response" ] || exit 1
    [ "$(echo "$response" | jq -r .error)" = 'file already exists' ] && exit 1
' _ {} \; \
-o -quit

Now it exits prematurely if curl fails, if the response is empty, or if .error matches.

bash how to error out when subshell error

3 Answers3