Sorting Blocks of Text Chronologically While Retaining Format

Question

I'm working on a BASH script that pulls email logs and puts the transactions into blocks of text such as:

   Dec 11 04:00:06 hostname postfix/pickup[5451]: 000A180464: uid=0 from=<root>
   Dec 11 04:00:06 hostname postfix/cleanup[8160]: 000A180464: message-id=<20151211090006.000A180464@domain.com>
   Dec 11 04:00:06 hostname postfix/qmgr[22497]: 000A180464: from=<root@domain.com>, size=59070, nrcpt=1 (queue active)
   Dec 11 04:00:06 hostname postfix/local[8168]: 000A180464: to=<root@localhost.localdomain>, orig_to=<root>, relay=local, delay=0.02, delays=0.01/0/0/0, dsn=2.0.0, status=sent (forwarded as 019B380405)
   Dec 11 04:00:06 hostname postfix/qmgr[22497]: 000A180464: removed

   Nov  8 12:36:24 hostname postfix/cleanup[11832]: 0016C82F49: message-id=<20151220050001.F2FD082F53@domain.com>
   Nov  8 12:36:24 hostname postfix/qmgr[1730]: 0016C82F49: from=<root@domain.com>, size=5206, nrcpt=1 (queue active)
   Nov  8 12:36:24 hostname postfix/pipe[11832]: 0016C82F49: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.01, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Nov  8 12:36:24 hostname postfix/qmgr[1730]: 0016C82F49: removed

   Nov  8 10:02:12 hostname postfix/cleanup[21832]: 000E583908: message-id=<20151108150211.F32D383919@domain.com>
   Nov  8 10:02:12 hostname postfix/qmgr[1920]: 000E583908: from=<root@domain.com>, size=5206, nrcpt=1 (queue active)
   Nov  8 10:02:12 hostname postfix/pipe[21838]: 000E583908: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.01, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Nov  8 10:02:12 hostname postfix/qmgr[1920]: 000E583908: removed

   Dec  5 16:00:19 hostname postfix/cleanup[22477]: 000FD8036A: message-id=<20151205210018.F324D80400@domain.com>
   Dec  5 16:00:19 hostname postfix/qmgr[1864]: 000FD8036A: from=<root@domain.com>, size=58511, nrcpt=1 (queue active)
   Dec  5 16:00:19 hostname postfix/pipe[22495]: 000FD8036A: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.02, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Dec  5 16:00:19 hostname postfix/qmgr[1864]: 000FD8036A: removed

   Dec 14 13:00:17 hostname postfix/cleanup[5127]: 001528050D: message-id=<20151214180016.F33048050C@domain.com>
   Dec 14 13:00:17 hostname postfix/qmgr[22497]: 001528050D: from=<root@domain.com>, size=58835, nrcpt=1 (queue active)
   Dec 14 13:00:17 hostname postfix/pipe[5129]: 001528050D: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.02, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Dec 14 13:00:17 hostname postfix/qmgr[22497]: 001528050D: removed

As you can the the groups are good, but the groups aren't chronological. When I try to sort them I loose the grouping and separation between them. What I would like is to have the output sort by date and time such as:

   Dec  5 16:00:19 hostname postfix/cleanup[22477]: 000FD8036A: message-id=<20151205210018.F324D80400@domain.com>
   Dec  5 16:00:19 hostname postfix/qmgr[1864]: 000FD8036A: from=<root@domain.com>, size=58511, nrcpt=1 (queue active)
   Dec  5 16:00:19 hostname postfix/pipe[22495]: 000FD8036A: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.02, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Dec  5 16:00:19 hostname postfix/qmgr[1864]: 000FD8036A: removed

   Nov  8 10:02:12 hostname postfix/cleanup[21832]: 000E583908: message-id=<20151108150211.F32D383919@domain.com>
   Nov  8 10:02:12 hostname postfix/qmgr[1920]: 000E583908: from=<root@domain.com>, size=5206, nrcpt=1 (queue active)
   Nov  8 10:02:12 hostname postfix/pipe[21838]: 000E583908: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.01, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Nov  8 10:02:12 hostname postfix/qmgr[1920]: 000E583908: removed

   Nov  8 12:36:24 hostname postfix/cleanup[11832]: 0016C82F49: message-id=<20151220050001.F2FD082F53@domain.com>
   Nov  8 12:36:24 hostname postfix/qmgr[1730]: 0016C82F49: from=<root@domain.com>, size=5206, nrcpt=1 (queue active)
   Nov  8 12:36:24 hostname postfix/pipe[11832]: 0016C82F49: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.01, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Nov  8 12:36:24 hostname postfix/qmgr[1730]: 0016C82F49: removed

   Dec 11 04:00:06 hostname postfix/pickup[5451]: 000A180464: uid=0 from=<root>
   Dec 11 04:00:06 hostname postfix/cleanup[8160]: 000A180464: message-id=<20151211090006.000A180464@domain.com>
   Dec 11 04:00:06 hostname postfix/qmgr[22497]: 000A180464: from=<root@domain.com>, size=59070, nrcpt=1 (queue active)
   Dec 11 04:00:06 hostname postfix/local[8168]: 000A180464: to=<root@localhost.localdomain>, orig_to=<root>, relay=local, delay=0.02, delays=0.01/0/0/0, dsn=2.0.0, status=sent (forwarded as 019B380405)
   Dec 11 04:00:06 hostname postfix/qmgr[22497]: 000A180464: removed

   Dec 14 13:00:17 hostname postfix/cleanup[5127]: 001528050D: message-id=<20151214180016.F33048050C@domain.com>
   Dec 14 13:00:17 hostname postfix/qmgr[22497]: 001528050D: from=<root@domain.com>, size=58835, nrcpt=1 (queue active)
   Dec 14 13:00:17 hostname postfix/pipe[5129]: 001528050D: to=<admin@domain.com>, orig_to=<root>, relay=plesk_virtual, delay=0.02, delays=0/0/0/0.02, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
   Dec 14 13:00:17 hostname postfix/qmgr[22497]: 001528050D: removed

Any help would be appreciated.

Why does Dec 5 come before Nov 8? – karakfa Jan 01 '16 at 20:53 — karakfa, Jan 01 '16 at 20:53

score 1 · Answer 1 · edited Jan 01 '16 at 21:46

1

Try this:

sort -s -k 1M -k 2n -k 3n logfile.txt

That is sort the file so that first column is a month (1M), second and third ones are numeric values (2n and 3n).

See also this discussion and man sort for more details.

edited Jan 01 '16 at 21:46

Gilles Quénot

173,512
41
224
223

answered Jan 01 '16 at 21:33

vrs

1,922
16
23

score 0 · Accepted Answer · answered Jan 01 '16 at 21:25

I guess this is what you want

$ sed '$a\\' log \
   | awk '!NF{c++; print k,c+1 "\t"; c++; next} \
             {k=$1 FS $2 FS $3; print k, c+1 "\t" $0}' \
   | sort -k1,1M -k2n -k3,3 -k4n \
   | cut -f2 | sed '$d'

   Nov  8 10:02:12 hostname postfix/cleanup[21832]
   Nov  8 10:02:12 hostname postfix/pipe[21838]: 0
   Nov  8 10:02:12 hostname postfix/qmgr[1920]: 00
   Nov  8 10:02:12 hostname postfix/qmgr[1920]: 00

   Nov  8 12:36:24 hostname postfix/cleanup[11832]
   Nov  8 12:36:24 hostname postfix/pipe[11832]: 0
   Nov  8 12:36:24 hostname postfix/qmgr[1730]: 00
   Nov  8 12:36:24 hostname postfix/qmgr[1730]: 00

   Dec  5 16:00:19 hostname postfix/cleanup[22477]
   Dec  5 16:00:19 hostname postfix/pipe[22495]: 0
   Dec  5 16:00:19 hostname postfix/qmgr[1864]: 00
   Dec  5 16:00:19 hostname postfix/qmgr[1864]: 00

   Dec 11 04:00:06 hostname postfix/cleanup[8160]:
   Dec 11 04:00:06 hostname postfix/local[8168]: 0
   Dec 11 04:00:06 hostname postfix/pickup[5451]: 
   Dec 11 04:00:06 hostname postfix/qmgr[22497]: 0
   Dec 11 04:00:06 hostname postfix/qmgr[22497]: 0

   Dec 14 13:00:17 hostname postfix/cleanup[5127]:
   Dec 14 13:00:17 hostname postfix/pipe[5129]: 00
   Dec 14 13:00:17 hostname postfix/qmgr[22497]: 0
   Dec 14 13:00:17 hostname postfix/qmgr[22497]: 0

Sorting Blocks of Text Chronologically While Retaining Format

2 Answers2