TCL: How do I group log messages to together based on date stamps

Question

I'm fairly new to TCL and need some assistance grouping log messages together based on date stamps from a Cisco router.

UPDATED: change in sample log. have discovered that there is an extra space when the DD is a single digit. e.g " 1"

Samplelog : show logging

THREADID: Feb  1 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Feb  1 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Feb  1 HH:MM:SS.SSS  : %TYPE-OF-VAIRBLE: OFFLINE
THREADID: Feb  1 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Feb  3 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: UP
THREADID: Feb  3 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: UP
THREADID: Feb  4 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Feb  4 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Mar 15 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: UP
THREADID: Mar 15 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Mar 15 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Mar 16 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: UP
THREADID: Mar 16 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN
THREADID: Mar 16 HH:MM:SS.SSS : %TYPE-OF-VAIRBLE: DOWN

Desired output

Feb 01 || 3 down
Feb 04 || 2 down
Mar 15 || 2 down
Mar 16 || 2 down

Updated with new code with new logic to try and address extra space, although this is not working as expected.

set y [read -nonewline [set f [open "tmp.txt" r]]];

array set counts [list];
foreach message [split $y "\n"] {
    # This gets the status, ie: DOWN/UP/OFFLINE.
    set status [lindex [split $message] end]; # will assign last field
    set mon [lindex [split $message] 1]; # number represents the filed number
    set day [lindex [split $message] 2];
    if {$day ==""} {
        set day [lindex [split $message] 3];
     } else {
        set day [lindex [split $message] 2]
    }
        # +number represents the counter
    if {[info exists counts($mon,$day,$status)]} {
            set counts($mon,$day,$status) [expr {$counts($mon,$day,$status)+1}] 
    } else {
            set counts($mon,$day,$status) 1
    }   
}

# sort based for down type events

puts "\n\n MMM DD || Cnt Status"
puts " ====================="

foreach count [lsort -increasing -unique [array names counts]] {

    foreach {mon day status} [split $count ","] { break; }
    if {$status =="down"} {
    puts " $mon $day || [set counts($count)] \t $status"
    }
}

if {[info exists f]} { close $f }

That grouping isn't based on date stamps. – potrzebie Mar 07 '13 at 01:19 — potrzebie, Mar 07 '13 at 01:19

score 1 · Answer 1 · answered Mar 07 '13 at 06:44

Just use an associative array — array (sic!) or dict (appeared in Tcl 8.5).

Before scanning, your array is empty, and as you scan the log file and parse lines of text from it (depending on the actual file format, this could be done with split or regexp or string range etc), you check to see if an entry matching your grouping criteria already exists in the array. If it does not, you insert some data structure to your array, populated by some initial value(s). If a matching entry exists, you update the data structure associated with it with the new data.

As to actual data structures... For arrays I would use two-element lists — with their elements holding the number of "up" and "down" events, respectively. dictionaries could be nested, so you could use this possibility.

Note that I don't have a clear idea of what your log file is really about, so may be the elements of your associative array should keep some other values, but the general idea still holds: it's a task for an associative array.

Thanks for the answer make sense but a lot of information to process and understand. — onxx, Mar 07 '13 at 17:53

Modul8 · Accepted Answer · 2013-03-07T11:03:43.190

1

Try this - you will need to change "file.txt" to your log file.

set y [read -nonewline [set f [open "file.txt" r]]];

array set counts [list];
foreach message [split $y "\n"] {
        # This gets the status, ie: DOWN/UP/OFFLINE.
        set status [lindex [split $message] end];
        set id [lindex [split $message] 3];
        if {[info exists counts($id,$status)]} {
                set counts($id,$status) [expr {$counts($id,$status)+1}]
        } else {
                set counts($id,$status) 1
        }
}

foreach count [lsort -increasing -unique [array names counts]] {
        foreach {name status} [split $count ","] { break; }
        puts "MMM $name || [set counts($count)] $status"
}

if {[info exists f]} { close $f }

The output for this (for the example log you gave) gives:

MMM 01 || 3 DOWN
MMM 01 || 1 OFFLINE
MMM 03 || 2 UP
MMM 04 || 2 DOWN

edited Mar 07 '13 at 11:03

answered Mar 07 '13 at 10:57

Modul8

46
2

Hi, Thanks for the example although it doesn't give me the desired output. it displays MMM instead of the month. Could you please also provide more comments to your code so I can understand how it works and tweak as necessary. – onxx Mar 07 '13 at 17:31
If I wanted to to only output log messages over a certain count how would I do that? e.g only counts over 3. output:: MMM 01 || 3 DOWN – onxx Mar 07 '13 at 22:15
In your second bit, the foreach count [lsort ...], add if {[set counts($count)] < 3} { continue; } – Modul8 Mar 08 '13 at 18:53
Thanks, one more question and I think its complete. I have found out that the log doesn't put "01" it actually does this " 1" adds an extra space when the day is a single digit. I have updated my question and code. is there a better way do process the extra space? – onxx Mar 11 '13 at 21:39
You could use [string trim $day] to remove any extra whitespace at the start :) – Modul8 Mar 14 '13 at 13:07

TCL: How do I group log messages to together based on date stamps

2 Answers2