How to add counter to find xargs

Question

So I have this code, to puke 5 files per script. But, I need a counter

find cobacoba -type f | xargs -n 5 bash -c 'script.sh $counter ${0} ${1} ${2} ${3} ${4}' bash

Script.sh:

#!/usr/bin/env bash

echo "Group $0: $1 $2 $3 $4 $5"

##This is just a simple example. The actual script will use each variable, including the counter, for further processing.
##So, really need all the variables being passed by "find | xargs"

Hoping Result:

Group 1: cobacoba/1.3 cobacoba/1.6 cobacoba/1.q cobacoba/1.5
Group 2: cobacoba/1.1 cobacoba/1.q2 cobacoba/1.q23 cobacoba/1.4
Group 3: cobacoba/1.2

What strategy can I use to create $counter ?

What's wrong with [Need counter on find and xarg combo](https://stackoverflow.com/questions/57450254/need-counter-on-find-and-xarg-combo) you selected an answer. Why two similar questions? — David C. Rankin, Aug 12 '19 at 02:36
I am still working on it. I thought his solution: AWK is only printout function. I am trying to google whether I can run bash script from inside AWK. I accepted his answer because his solution answer my question but not exactly what I need. I need to run a bash script — Call Me Hi Hi, Aug 12 '19 at 02:40

Léa Gris · Answer 1 · 2019-08-12T11:45:04.417

Since the right side of a | pipe is a sub-shell, it can not persist or update a variable. It is not possible to use a counter in a variable like you do.

Fortunately Bash allow you to reverse the operation and feed the main shell (here a while loop) with the output of a command running in a sub-shell. Since we update our variables within the main shell, it works like that:

#!/usr/bin/env bash

group=1           # Group counter
files_per_group=4 # How many files per group
filescount=0      # Files counter
newline=''        # Newline code used between groups

# loop reading all null delimited files names returned by find -print0
while read -d '' -r file; do
  # If count of file is a multiple of files per group, then start a new group
  if [ $((filescount % files_per_group)) -eq 0 ]; then
    printf '%sGroup %d:' "$newline" "$group"
    # Increment group counter for upcoming group
    group=$((group + 1))
    newline=$'\n' # To separate next group in another line
  fi
  # Print space delimited file-name
  printf ' %s' "$file"
  # Increment the files counter
  filescount=$((filescount + 1))
done < <(
  # feed the whole while loop with the output of find
  find cobacoba -type f -print0
)
echo

Solution with building parameters to call a bash script:

#!/usr/bin/env bash

# Dummy bashscript as a function to test call with parameters
bashscript() {
  printf 'Called bashscript.sh\n'
  printf 'group %s\n' "$1"
  shift
  printf '%d files:\n' "$#"
  printf ' %s' "$@"
  printf $'\n\n'
}

group=1             # Group counter
files_per_group=5   # How many files per group
filescount=0        # Files counter
bashscriptparams=() # Arguments for the bash scripts

# loop reading all null delimited files names returned by find -print0
while read -d '' -r file; do
  # If count of file is a multiple of files per group, then start a new group
  if [ $((filescount % files_per_group)) -eq 0 ]; then
    #printf '%sGroup %d:' "$newline" "$group"
    # Set the group number as first param of the bash script
    bashscriptparams=("$group")
    # Increment group counter for upcoming group
    ((group++))
  fi
  # Add the file as next parameter of the bash script
  bashscriptparams+=("$file")
  # Print space delimited file-name
  #printf ' %s' "$file"
  # If last file of group, then group is complete
  if [ $((filescount % files_per_group)) -eq $((files_per_group - 1)) ]; then
    # Launch the bash script with its arguments (group file_1 .. file_n)
    bashscript "${bashscriptparams[@]}"
  fi
  # Increment the files counter
  ((filescount++))
done < <(
  # feed the whole while loop with the output of find
  find cobacoba -type f -print0
)
# If we reach here with an incomplete files group
if [ $((filescount % files_per_group)) -le $((files_per_group - 1)) ]; then
  # Launch the bash script with its incomplete files arguments (group file_1 .. file_n)
  bashscript "${bashscriptparams[@]}"
fi

Output:

Called bashscript.sh
group 1
5 files:
 cobacoba/1.q2 cobacoba/1.3 cobacoba/1.1 cobacoba/1.5 cobacoba/1.6

Called bashscript.sh
group 2
4 files:
 cobacoba/1.2 cobacoba/1.4 cobacoba/1.q cobacoba/1.q23

Thanks, but with this script, what will happen if there are less than 5 files, like 3? I think it will skip the 3 files? — Call Me Hi Hi, Aug 12 '19 at 02:53
Mybe you can try and see for yourself. Because I tested my own answer. — Léa Gris, Aug 12 '19 at 02:55
ahhh, yes it's working :) But this is manipulating "print" to get the result. I would like to use the variables. How do I extract them: the group counter and the files (respectively to their group) so that I can use them in another bash script? — Call Me Hi Hi, Aug 12 '19 at 03:09
Added the solution with calling the bash script building parameters dynamically. @CallMeHiHi — Léa Gris, Aug 12 '19 at 11:45

score 0 · Accepted Answer · answered Aug 13 '19 at 15:29

0

There's no need to use find or xargs if all you want to do is recursively walk a directory:

i=0
shopt -s globstar
for f in cobacoba/**; do
  [[ -f $f ]] || continue
  (( i > 5 )) && wait
  script.sh "$i" "${0}" "${1}" "${2}" "${3}" "${4}" "$f" &
  (( i++ ))
done

answered Aug 13 '19 at 15:29

chepner

497,756
71
530
681

what does `[[ -f $f ]] || continue` mean? – Call Me Hi Hi Aug 14 '19 at 23:50
If `$f` is not a regular file, continue to the next iteration of the loop. It simulates the `-type f` primary in `find`. – chepner Aug 14 '19 at 23:55

How to add counter to find xargs

2 Answers2