UNIX(AIX) script to process a file using only awk or other file processing utilities

Question

I have a task to write a script that will filter an input from an MQ runmqsc command and redirect the output into another file. I have been working around using many other Linux commands piped together and it seems to work just fine in Linux, but my manager needs to run the script on an AIX system, so UNIX operating system. I was made aware that many commands that run fine on Linux or get the job done in Linux will produce a totally different output on UNIX or UNIX-based systems. The output from the runmqsc command looks like this:

5724-H72 (C) Copyright IBM Corp. 1994, 2009.  ALL RIGHTS RESERVED.
Starting MQSC for queue manager CNUMQ02B.


     1 : DISPLAY QLOCAL(*) CURDEPTH
AMQ8409: Display Queue details.
   QUEUE(ADEXA.AOM.REPLY.MR.QL)            TYPE(QLOCAL)
   CURDEPTH(0)                          
AMQ8409: Display Queue details.
   QUEUE(ADEXA.AOM.REPLY.QL)               TYPE(QLOCAL)
   CURDEPTH(0)                          
AMQ8409: Display Queue details.
   QUEUE(ADEXA.ERROR.QL)                   TYPE(QLOCAL)
   CURDEPTH(0)                          
AMQ8409: Display Queue details.
   QUEUE(ADEXA.FACT.OUT.QL)                TYPE(QLOCAL)
   CURDEPTH(0)
AMQ8409: Display Queue details.
   QUEUE(ADW.REMAN.XREF.ERR.QL)            TYPE(QLOCAL)
   CURDEPTH(14)
AMQ8409: Display Queue details.
   QUEUE(SAPNA.MESS.CRITICAL.CLASS.RESUME.QL)
   TYPE(QLOCAL)                            CURDEPTH(123)
One MQSC command read.
No commands have a syntax error.
All valid MQSC commands were processed.

What I basically need to do is only display the name of the queue, a whitespace, and then the queue depth on the same line, with no trailing whitespaces and no newline characters at the beginning or end of file, so that it will resemble a csv file with the whitespace as the separator. I also need to filter out queues that have a queue depth equal to 0, so the output will look like this:

ADW.REMAN.XREF.ERR.QL 14

As I said I tried many commands on Linux, but I have a lack of knowledge of what commands and flags actually work more or less the same on UNIX and Linux, and my manager wants this today, so if by any chance you read this I ask that you at least guide me what to use to try working it out :) Thanks.

This is what I wrote in Linux:

head -n -3 "$1" | 
tail -n +6 | 
sed '/AMQ8409: Display Queue details./d' | 
sed 's/TYPE(QLOCAL)//g' | 
tr -d ' \t\r\f' | 
awk 'NR%2{printf "%s ",$0;next;}1' | 
sed '/CURDEPTH(0)/d' | 
awk '{gsub(/QUEUE(/, ""); gsub(/CURDEPTH(/, ""); gsub(/)/, ""); print}' |
sort -nk2 

Answer 1

Try this simpler command..

sed -n -e 's/.*QUEUE(\([^)]*\)).*/\1/p' -e 's/.*CURDEPTH(\([0-9]*\)).*/\1/p' \
   | paste -d ' ' - - \
   | grep -v ' 0$'

Answer 2

$ awk '/QUEUE|CURDEPTH\(/ {                            # on matching records
           gsub(/^[^(]+\(|).*$/,"");                   # remove unwanted parts
           printf "%s%s", $0, ($0~/^[0-9]+$/?ORS:OFS)  # print ORS after depth
       }' file
ADEXA.AOM.REPLY.MR.QL 0
ADEXA.AOM.REPLY.QL 0
ADEXA.ERROR.QL 0
ADEXA.FACT.OUT.QL 0
ADW.REMAN.XREF.ERR.QL 14

Answer 3

Here's one that doesn't use gsub:

awk -F "[()]" '/QUEUE/ {quename=$2} /CURDEPTH\(/ {print quename, $2} '

Answer 4

Rather than use "head" and "tail" and "sed" to remove unwanted lines, use grep to take the ones you do want. I am not sure which of these options will work on AIX, but they all should I believe.

grep -B 1 CURDEPTH $1
grep -A 1 QUEUE $1
grep -e QUEUE -e CURDEPTH $1

Next, join and format the lines:

sed 's#.*QUEUE(\([^)]*\)).*\n.*CURDEPTH(\([0-9]*\)).*#\1 \2#'

If that doesn't work properly, try using '\r' instead of '\n'.

Lastly, remove all lines with a depth of 0 (2 options):

grep -v " 0$"
sed '# 0$#d'

I'm not very familiar with this form of sed - I pinched it from your code example - but it looks simple so should work.

All these commands are about the most basic form you can get so should work equally well on any *nix system. Hopefully. The initial grep and the sed are the highest points of risk.

Answer 5

This works for me on AIX to get the output you require, this uses the IBM MQ's WHERE clause to restrict the output to queues with a CURDEPTH greater than 0:

echo "DIS QL(*) WHERE(CURDEPTH GT 0)" | runmqsc <qmgr> | sed -e 's/\((.*)\)  *\(.*\)/\1\
\2/g' -e 's/^ *\([A-Z][A-Z]*[(][^)][^)]*[)]\)/\1/g' -e 's/^\(AMQ[0-9][0-9]*:\).*$/\1/g' | awk -F '[()]' -v OFS=" " 'function printValues() { if ("QUEUE" in p) { print p["QUEUE"], p["CURDEPTH"] } } /AMQ8409:/ { printValues(); for (i in p) { delete p[i] }; next } { p[$1] = $2 } END { printValues() }'

Note that above is two lines, the \ at the end of the first line allows the sed command to replace with an embedded new line. The awk printValues syntax is from @mike.dld answer to this post "Print part of an array or a file and sort in shell". I had to replace his delete p with the for (i in p) { delete p[i] } since AIX awk did not support the other syntax.

If you want to run with a file as input use the following:

cat file | sed -e 's/\((.*)\)  *\(.*\)/\1\
\2/g' -e 's/^ *\([A-Z][A-Z]*[(][^)][^)]*[)]\)/\1/g' -e 's/^\(AMQ[0-9][0-9]*:\).*$/\1/g' | awk -F '[()]' -v OFS=" " 'function printValues() { if ("QUEUE" in p) { print p["QUEUE"], p["CURDEPTH"] } } /AMQ8409:/ { printValues(); for (i in p) { delete p[i] }; next } { p[$1] = $2 } END { printValues() }' | grep -v ' 0$'

It is easy to add any number of attributes to the output. If you want to expand it in the future you just add more entries to the awk command for each attribute required like this: , p["ATTR1"], p["ATTR2"]

Using MAXDEPTH as an example:

echo "DIS QL(*) WHERE(CURDEPTH GT 0)" | runmqsc <qmgr> | sed -e 's/\((.*)\)  *\(.*\)/\1\
\2/g' -e 's/^ *\([A-Z][A-Z]*[(][^)][^)]*[)]\)/\1/g' -e 's/^\(AMQ[0-9][0-9]*:\).*$/\1/g' | awk -F '[()]' -v OFS=" " 'function printValues() { if ("QUEUE" in p) { print p["QUEUE"], p["CURDEPTH"], p["MAXDEPTH"] } } /AMQ8409:/ { printValues(); for (i in p) { delete p[i] }; next } { p[$1] = $2 } END { printValues() }'

Note that this answer and probably others will have an issue if the attribute itself has embedded parenthesis, this is possible in the DESCR field of the queue and is true of various other fields on other objects types. One common example is the CONNAME attribute of a CHANNEL. The CONNAME uses the format hostname(port) so is displayed as CONNAME(hostname(port)).

Answer 6

With awk, that can be done relatively easily:

runmqsc "display qlocal(*) curdepth" | awk '
    /QUEUE\(/ {
        gsub(/^[^\(]*\(/, "", $0);
        gsub(/\).*$/, "", $0);
        qname = $0;
    }
    /CURDEPTH\(/ {
        gsub(/^[^\(]*\(/, "", $0);
        gsub(/\).*$/, "", $0);
        print qname" "$0;
    }'

This basically just extracts the queue name from lines containing QUEUE( and stores them for later.

When it finds lines containing CURDEPTH(, it extracts the depth the same way then outputs that along with the stored queue name.

The two gsub calls in each section look like "black magic" but they're easy to understand once you know what they mean. The first finds a pattern at line start consisting of the largest number of non-( characters followed by a ( character, then replaces that with nothing.

The second is similar but it removes the largest sequence of ") followed by any characters to end of line".