awk | merge line on the basis of field matching

Question

I need help with following:

Input file:

abc message=sent session:111,x,y,z
pqr message=receive session:111,4,5,7
abc message=sent session:123,x,y,z
pqr message=receive session:123,4,5,7
abc message=sent session:342,x,y,z
abc message=sent session:589,x,y,z
pqr message=receive session:589,4,5,7

Output file:

abc message=sent session:111,x,y,z, pqr message=receive session:111,4,5,7
abc message=sent session:123,x,y,z, pqr message=receive session:123,4,5,7
abc message=sent session:342,x,y,z, NOMATCH
abc message=sent session:589,x,y,z, pqr message=receive session:589,4,5,7

Notes:

If you see in source file, for every "sent" message there is "receive"
only for session=342 there is no receive
session is unknow, can't be hardcoded
So merge only those sent and receive where we have matching session number

does the message=receive always follow the message=sent immediately, like this example? — jkerian, Feb 13 '13 at 05:58
not always.. if there is "message=sent" and "message=receive" for same session then only combine — Vipin Choudhary, Feb 13 '13 at 06:00
also if you see then we have 2 message=sent (one after another) in an example, which means I need to skip one sent and continue with next line — Vipin Choudhary, Feb 13 '13 at 06:01
do you have to awk and sed? Doing this in perl would be easy. With awk and sed it could get ugly. — Red Cricket, Feb 13 '13 at 06:10

Steve · Answer 1 · 2013-02-13T06:19:32.403

Here's one way using awk. Run like:

awk -f script.awk file

Contents of script.awk:

{
    x = $0

    gsub(/[^:]*:|,.*/,"")

    a[$0] = (a[$0] ? a[$0] "," FS : "") x
    b[$0]++
}

END {
    for (i in a) {
        print (b[i] == 2 ? a[i] : a[i] "," FS "NOMATCH") | "sort"
    }
}

Results:

abc message=sent session:111,x,y,z, pqr message=receive session:111,4,5,7
abc message=sent session:123,x,y,z, pqr message=receive session:123,4,5,7
abc message=sent session:342,x,y,z, NOMATCH
abc message=sent session:589,x,y,z, pqr message=receive session:589,4,5,7

Alternatively, here's the one-liner:

awk '{ x = $0; gsub(/[^:]*:|,.*/,""); a[$0] = (a[$0] ? a[$0] "," FS : "") x; b[$0]++ } END { for (i in a) print (b[i] == 2 ? a[i] : a[i] "," FS "NOMATCH") | "sort" }' file

Note that you can drop the pipe to sort if you don't care about sorted output. HTH.

Hello Steve.. could you please explain the logic and why we are putting sort at the end? — Vipin Choudhary, Feb 14 '13 at 09:56

Guru · Accepted Answer · 2013-02-13T07:00:28.783

1

Another way:

awk -F "[:,]"  '/=sent/{a[$2]=$0;}/=receive/{print a[$2], $0;delete a[$2];}END{for(i in a)print a[i],"NO MATCH";}' file

Results:

abc message=sent session:111,x,y,z pqr message=receive session:111,4,5,7
abc message=sent session:123,x,y,z pqr message=receive session:123,4,5,7
abc message=sent session:589,x,y,z pqr message=receive session:589,4,5,7
abc message=sent session:342,x,y,z NO MATCH

When the send record is encountered, it is store in the array with the session id as the index. When the receive record is encountered, the send record is fetched from the array and printed along with receive record. Also, sent records are removed from array as and when receive records are received. At the END, all the remaining records in the array are printed as NO MATCH.

edited Feb 13 '13 at 07:00

answered Feb 13 '13 at 06:29

Guru

16,456
2
33
46

Thanks a lot for this.. but I am not able to understand the logic.. could you please explain it – Vipin Choudhary Feb 13 '13 at 06:54
Hello again Guru.. I have one more query.. is it possible to print output in a sequence in its order? for example NO MATCH should be print in between after session 123 – Vipin Choudhary Feb 14 '13 at 09:52
If I'll reprase it then it should be like this: For every "=sent", search for "=receive" in **immediate** NEXT LINE ONLY for same session number
So merge only those sent and receive where we have matching session number ELSE print the sent as it is in a sequence
– Vipin Choudhary Feb 14 '13 at 12:27

awk | merge line on the basis of field matching

2 Answers2

Linked