0

I'm trying to filter the following log file:

+---------+---------+---------+---------+---------+---------+---------+----
.Logon hostname/username,

 *** Logon successfully completed.
 *** Teradata Database Release is 14.00.06.05
 *** Teradata Database Version is 14.00.06.05
 *** Transaction Semantics are BTET.
 *** Session Character Set Name is 'ASCII'.

 *** Total elapsed time was 1 second.

+---------+---------+---------+---------+---------+---------+---------+----
select current_timestamp as started_test;

 *** Query completed. One row found. One column returned.
 *** Total elapsed time was 1 second.

                    started_test
--------------------------------
2014-10-06 17:44:39.220000+00:00

+---------+---------+---------+---------+---------+---------+---------+----
select * from database.view sample 2;

 *** Query completed. 2 rows found. 41 columns returned.
 *** Total elapsed time was 2 seconds.


select current_timestamp as finished_test;

 *** Query completed. One row found. One column returned.
 *** Total elapsed time was 1 second.

                   finished_test
--------------------------------
2014-10-06 17:44:41.330000+00:00

with this logstash filter

input{
        file {
                path => "/home/iv41/perfmon.log"
        }
        stdin {}
}

filter {
        grok{
                match => ["message", "%{/\s+started_test/:start_time} START id: (?<task_id>.*)"]
                add_tag => ["testStarted"]
        }

        grok{
                match => ["message", "%{/\s+finished_test/:end_time} END id: (?<task_id>.*)"]
                add_tag => ["testEnded"]
        }

        if [start_time] != "/\s+started_test/"{
                if [end_time] != "/\s+finished_test/"{
                        drop {}
                }
        }

        elapsed {
                start_tag => "testStarted"
                end_tag => "testEnded"
                unique_id_field => "task_id"
        }
}

output{
        stdout {}
}

I think there may be issues with my regex's and task ids.

Essentially, I'm trying to pull out the time it takes between "started_test" and "finished_test". Does anyone know a better way of doing it? or know where my code is out?

arbitrage_junkie
  • 47
  • 6
  • The `%{NAMED_PATTERN:capture_name}` syntax is for named captures. You will also need to use the multiline codec or filter to combine the lines together so that you can match propertly. – Alcanzar Oct 06 '14 at 18:03
  • thanks Alcanzar. It matches using the NAMED_PATTERN syntax as you noted, but I don't know how to tag it on the condition that the word is 'started' or 'finished'. I tried an If statement in the grok but it just fails with an error saying the if isn't expected there.. – arbitrage_junkie Oct 06 '14 at 19:22
  • you don't want to use `!=` in there.. you want to use `!~` (the docs show `regexp: =~, !~`) – Alcanzar Oct 06 '14 at 19:25

0 Answers0