1

How do I use GNU Parallel to run shellcheck on all *.sh with a wrapper function?

This attempt has the correct target files, but Parallel seems to struggle with the subshell.

lint:sh() {
  local output
  targets=$(find . -name '*.sh' -exec echo {} \;)
  output=$(parallel --no-notice shellcheck ::: "$targets")
  results $? "$output"
}
M. Becerra
  • 639
  • 6
  • 21
Tom
  • 981
  • 11
  • 24
  • 1
    I think you want `targets` to be an array. `targets=($(find ...))` then pass `”${targets[@]}` to **GNU Parallel** – Mark Setchell Jun 26 '18 at 06:18
  • 2
    `find ... -exec echo {} \;` seems overly verbose - try omitting `-exec` and everything after. – Mark Setchell Jun 26 '18 at 06:23
  • When using an array (verbose is not he current issue) `mapfile -t targets < <(find . -name '*.sh' -exec echo {} \;) output=$(parallel --no-notice shellcheck ::: "${targets[@]}")` i get a non-zero exit but in the subshell not, lint:sh - results is not invoked – Tom Jun 26 '18 at 16:40
  • And its it is invoking the targets rather than passing them to shellcheck – Tom Jun 26 '18 at 16:56
  • I don't think a colon is permitted in a function name, btw. – Mark Setchell Jun 26 '18 at 16:58
  • I don't think the : name space delim is the issue in this post but I can use _ instead, especially if that is more typical for shell scripts – Tom Jun 26 '18 at 17:04

2 Answers2

1

Use the following approach:

lint:sh() {
  local output
  output=$(find . -name "*.sh" -print0 | parallel -q0 --no-notice shellcheck)
  ...
}
RomanPerekhrest
  • 88,541
  • 4
  • 65
  • 105
  • Thanks, but this bites both problems of eating the error codes and the output. – Tom Jun 26 '18 at 16:35
  • @Tom, the goal was to show you a working GNU parallel approach. But `results` is your custom "feature". Consider my approach to only this line `output=$(find . -name "*.sh" -print0 | parallel -q0 --no-notice shellcheck)` – RomanPerekhrest Jun 26 '18 at 16:40
  • That line exits 1 and kills the wrapper script - so results is never invoked. I also can not `echo $output` instead of ` results ...` Using std in vs redirect with ::: does not seem to be the issue. – Tom Jun 26 '18 at 17:01
0

due to different flaws in bash the safiest way may be

targets=()
while read -r -d '' file; [[ $file ]]; do
    targets+=("$file");
done < <(find . -name '*.sh' -print0)

also because targets type was changed to array

output=$(parallel --no-notice shellcheck ::: "${targets[@]}")
Nahuel Fouilleul
  • 18,726
  • 2
  • 31
  • 36
  • 1
    Using bash's globstar `**/*.sh` would be way easier. – Socowi Jun 26 '18 at 10:28
  • @Socowi, but would present a serious drawback because `**` would expand in memory and would be much slower. – Nahuel Fouilleul Jun 26 '18 at 12:31
  • 1
    Slower, yes (on my system by factor 3.5 for `/**/*.so`) but the memory argument doesn't count here. In both cases, all files are stored in an array which is stored in memory. – Socowi Jun 26 '18 at 13:04