how to extract data from trace files using awk and xxd

Question

I am trying to extract data from a trace files and calculate the sum. For a single file, the following command works :

   cat avg.txt| gawk '{T+=$1} END {print T "\n"}'

But when I try using it with xargs command, as I have to do the same operation on multiple fils,I get a syntax error.

     ls *avg* |  xargs -i sh -c " cat {} | gawk '{T+=$1} END {print T "\n"}'"

gawk: {T+=} END {print T n}

gawk: ^ syntax error

Can someone tell me what is the problem?

change your sh invocation to `sh -vxc "...` so you can see that is being procssed by the shell. Also, escape your awk `\$1`. Good luck. — shellter, Jun 06 '14 at 19:36

score 0 · Answer 1 · answered Jun 06 '14 at 19:51

0

xargs is adding a lot of complexity:

for file in *avg*; do
    gawk '{T+=$1} END {print FILENAME ": "T}' "$file"
done

The main problem with your code was the outer double quotes that allowed $1 to be expanded as a shell variable .

answered Jun 06 '14 at 19:51

glenn jackman

But what if we wanted to run over thousands of files, where `xargs -P8` would help, but then the total would get spread over those multiple invocations of `awk`. Would piping to `bc` be the only option then ? – Amit Naidu Jun 16 '20 at 00:30

score 0 · Answer 2 · answered Jul 17 '14 at 19:29

0

With GNU Parallel you would do this:

sum() {
  gawk '{T+=$1} END {print FILENAME ": "T}' $1
}
export -f sum
parallel sum ::: *avg*

answered Jul 17 '14 at 19:29

Ole Tange

2 Answers2