Unable to get GROK pattern to match even though it tests properly

Question

I am a new Logstash user and I am starting to write some grok rules to parse out my asa log files. I have a few rules triggering properly, and I am unable to get one to parse properly event though I test it out in grok debugger and it always tests properly. This event will always have the _grokparsefailure flag.

Here is an event:

<166>:Feb 26 23:44:14 PST: %ASA-session-6-305012: Teardown dynamic TCP translation from inside:192.168.1.45/53838 to outside:71.110.113.180/53838 duration 0:00:30

And my grok pattern:

<%{POSINT:syslog_pri}>:%{CISCOTIMESTAMP:timestamp} PST: %ASA-session-6-305012: Teardown dynamic TCP translation from %{WORD:source_interface_name}:%{IP:source_ip}/%{POSINT:source_port} to %{WORD:destination_interface_name}:%{IP:destination_ip}/%{POSINT:destination_port} duration (?<translation_duration>\d+:\d+:\d+)

My filter set is below:

filter {
        grok {
                match   => ["message", "<%{POSINT:syslog_pri}>:%    {CISCOTIMESTAMP:timestamp} PST: %ASA-session-6-305011: Built dynamic TCP translation from %{WORD:source_interface_name}:%{IP:source_ip}/%{POSINT:source_port} to %{WORD:destination_interface_name}:%{IP:destination_ip}/%{POSINT:destination_port}" ]

            match   => ["messgae", "<%{POSINT:syslog_pri}>:%{CISCOTIMESTAMP:timestamp} PST: %ASA-session-6-305012: Teardown dynamic TCP translation from %{WORD:source_interface_name}:%{IP:source_ip}/%{POSINT:source_port} to %{WORD:destination_interface_name}:%{IP:destination_ip}/%{POSINT:destination_port} duration (?<translation_duration>\d+:\d+:\d+)" ]

            match   => ["message", "<%{POSINT:syslog_pri}>:%{CISCOTIMESTAMP:timestamp} PST: %ASA-session-6-305011: Built dynamic UDP translation from %{WORD:source_interface_name}:%{IP:source_ip}/%{POSINT:source_port} to %{WORD:destination_interface_name}:%{IP:destination_ip}/%{POSINT:destination_port}" ]

            match   => ["message", "<%{POSINT:syslog_pri}>:%{CISCOTIMESTAMP:timestamp} PST: %ASA-session-6-305012: Teardown dynamic UDP translation from %{WORD:source_interface_name}:%{IP:source_ip}/%{POSINT:source_port} to %{WORD:destination_interface_name}:%{IP:destination_ip}/%{POSINT:destination_port} duration (?<translation_duration>\d+:\d+:\d+)" ]

    }

    geoip {
            source  => "source_ip"
    }

    geoip {
            source  => "destination_ip"
    }

Thanks for any guidance.

score 0 · Answer 1 · answered Feb 28 '15 at 16:29

try using the built-in grok filters instead of the named capture at the end like so:

<%{POSINT:syslog_pri}>:%{CISCOTIMESTAMP:timestamp} PST: %ASA-session-6-305012: Teardown dynamic TCP translation from %{WORD:source_interface_name}:%{IP:source_ip}/%{POSINT:source_port} to %{WORD:destination_interface_name}:%{IP:destination_ip}/%{POSINT:destination_port} duration %{NONNEGINT:dur_hour}:%{NONNEGINT:dur_min}:%{NONNEGINT:dur_sec}

You can also try to create a simple test.conf that simply uses as input:

stdin{}

and sets the output to:

output { stdout { codec => rubydebug } }

If you do a logstash -f test.conf < [your test data] it should give you additional info on what is going on.

Unable to get GROK pattern to match even though it tests properly

1 Answers1