Awk command print max min values from log

Question

I am doing grep with success flag S. $11 is the service type, $12 is either S(success) or F(failure), and $14 is elapsed time. Below command I am getting average elapsed time.

bash-3.2$ grep 'EXSTAT|' ivrbroker.log | grep '|F|' |
> /usr/xpg4/bin/awk -F"|" '{a[$11]++;c[$11]+=$14}
>    END{for(b in a){print b"," a[b]","c[b]/a[b]}}'
QCPE,2,276.5
bash-3.2$ grep 'EXSTAT|' ivrbroker.log|grep '|F|'       
EXSTAT|IVR|2014|11|17|14|43|57|1086|SRQCPE952|QCPE|F|100|349
EXSTAT|IVR|2014|11|17|15|35|51|1092|SRQCPE741|QCPE|F|100|204

But now I am looking for max elapsed time and min elapsed time like below.

QCPE,2,276.5,349,204

bash-3.2$ grep '|F|' ivrbroker.log
EXSTAT|IVR|2014|11|17|14|43|57|1086|SRQCPE952|QCPE|F|100|349
EXSTAT|IVR|2014|11|17|15|35|51|1092|SRQCPE741|QCPE|F|100|204
bash-3.2$ awk -F'|' 'BEGIN { OFS="," }
    /EXSTAT\|/ && /\|F\|/ { a[$11]++; c[$11] += $14;
        if (a[$11]==1) { max[$11]=$14; min[$11]=$14; }
        if($14 > max[$11]) max[$11]=$14;  if($14 < min[$11]) min[$11]=$14; }
    END { for(b in a) print b, a[b], c[b]/a[b], max[b], min[b] }' ivrbroker.log
,204,2,0,349
bash-3.2$ /usr/xpg4/bin/awk -F'|' 'BEGIN { OFS="," }                                                                                                                                                     
    /EXSTAT\|/ && /\|F\|/ { a[$11]++; c[$11] += $14;
       if (a[$11]==1) { max[$11]=$14; min[$11]=$14; }
       if($14 > max[$11]) max[$11]=$14;  if($14 < min[$11]) min[$11]=$14; }
    END { for(b in a) print b, a[b], c[b]/a[b], max[b], min[b] }' ivrbroker.log
,204,2,276.5,349
bash-3.2$

The `/usr/xpg4/bin/awk` path looks like you are not at all on Linux. Is the tag [tag:linux] incorrect? If so, which platform are you on? — tripleee, Nov 20 '14 at 07:57
bash-3.2$ uname -a SunOS utibcouat1 5.10 Generic_150400-13 sun4u sparc SUNW,Sun-Fire-V440 OS is sun solaris 5.10. — Guru, Nov 20 '14 at 08:05
I don't have easy access to `sawk` on Linux. If you can switch to `gawk` I believe that should fix any remaining compatibility issues. — tripleee, Nov 20 '14 at 08:35
Getting some syntax error for updated answer. I have edited and posted output for reference. — Guru, Nov 20 '14 at 09:12
Unfortunately, I have no access to a SysV `awk` to test on. The script works fine on `mawk`, `nawk`, and `gawk`. The problem seems to be with the expression `!($11 in max)` -- I have tried to devise a workaround, but I have no idea whether it will work. I updated my answer once more. — tripleee, Nov 20 '14 at 10:15
@Guru saying `Getting some syntax error` is **MUCH** less useful if you'd like us to help you debug it than `Getting THIS syntax error` and copy/pasting the actual syntax error after it. — Ed Morton, Nov 20 '14 at 13:40
You have a script that simply does the equivalent of `print a,b,c` and the output you show from that script is `,c,b`. Clearly you have a problem beyond the syntax of the script. Maybe you used a WIndows editor to create it and so it contains control characters? Maybe your input file contains control characters? Whatever it is you're going to have to debug it because there is nothing in this thread that would give anyone else any clue what is going on. The script as posted will NOT produce the output you show given the input you show, nor will it produce a syntax error on any modern awk. — Ed Morton, Nov 20 '14 at 14:05

tripleee · Answer 1 · 2014-11-20T10:16:51.617

You need to add two more array variables to keep track of the min and max.

While you are at it, get rid of the grep | awk antipattern.

awk -F'|' 'BEGIN { OFS="," }
    /EXSTAT\|/ && /\|F\|/ { a[$11]++; c[$11] += $14;
        if (!($11 in max)) max[$11]=min[$11]=$14;
        if($14 > max[$11]) max[$11]=$14;  if($14 < min[$11]) min[$11]=$14; }
    END { for(b in a) print b, a[b], c[b]/a[b], max[b], min[b] }' ivrbroker.log

Looks like maybe the first condition should be $1 == "EXSTAT" && $12 == "F" for slightly improved legibility and precision. Maybe also rename a to count and c to sum.

The above script works on Linux, but apparently not on SunOS / SysV / XPG4 Awk. Perhaps try this minor modification:

awk -F'|' 'BEGIN { OFS="," }
    /EXSTAT\|/ && /\|F\|/ { a[$11]++; c[$11] += $14;
        if (a[$11]==1) { max[$11]=$14; min[$11]=$14; }
        if($14 > max[$11]) max[$11]=$14;  if($14 < min[$11]) min[$11]=$14; }
    END { for(b in a) print b, a[b], c[b]/a[b], max[b], min[b] }' ivrbroker.log

I have updated the answer in the meantime. My original answer works for the sample data you posted (2 lines); if the example is not representative, please update it. — tripleee, Nov 20 '14 at 07:55
Your answer is completely portable and will work on all modern awks. If the OP is getting a syntax error then either he didn't copy/paste it correctly or he's trying to run old, broken awk (/bin/awk on Solaris) on it. — Ed Morton, Nov 20 '14 at 13:55

Awk command print max min values from log

1 Answers1