0

I'm setting up Logstash (ELK) as our central logging server and so far have had good luck with simple filters, but this latest one for our switches isn't coming as easily. Here's what a typical line looks like:

<179>12600: [syslog@9 s_id =\"SWITCH1:5143\"]: <ios-log-msg><facility>LINK</facility><severity>3</severity><msg-id>UPDOWN</msg-id><time>Jul 15 09:03:04</time><args><arg id=\"0\">GigabitEthernet1/0/32</arg><arg id=\"1\">up</arg></args></ios-log-msg>

Here's one of the patterns I'm working on:

<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA} s_id=(?:.{2})%{HOSTNAME:hostname}:5143(?:.{3}): <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg></args></ios-log-msg>

There's two key issues that I'm hung up on:

  1. The [syslog@9 s_id =\"BRD-STACK:5143\"] section contains the hostname of the switch. Everything else is static information and I want to discard (non-bold).
  2. Towards the end of the entry the "args" section can have a variable number of "arg" elements. This one has 2, and I've seen 1 and 3 as well depending on the message. I need the information contained in each of those.

Any ideas?

For what it's worth, here's the entire filter I was last playing with:

        grok {
        match => [
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA} s_id=(?:.{2})%{HOSTNAME:hostname}:5143(?:.{3}): <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg><arg id=\"2\">%{DATA:arg_2}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA} s_id=(?:.{2})%{HOSTNAME:hostname}:5143(?:.{3}): <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA} s_id=(?:.{2})%{HOSTNAME:hostname}:5143(?:.{3}): <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA} s_id=(?:.{2})%{HOSTNAME:hostname}:5143(?:.{3}): <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args>%{DATA:args}</args></ios-log-msg>"
        ]
    }
ceskib
  • 140
  • 1
  • 8

1 Answers1

0

And the magic combination is:

        match => [
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA}\"%{HOSTNAME:hostname}:5143\"\]: <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg><arg id=\"2\">%{DATA:arg_2}</arg><arg id=\"3\">%{DATA:arg_3}</arg><arg id=\"4\">%{DATA:arg_4}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA}\"%{HOSTNAME:hostname}:5143\"\]: <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg><arg id=\"2\">%{DATA:arg_2}</arg><arg id=\"3\">%{DATA:arg_3}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA}\"%{HOSTNAME:hostname}:5143\"\]: <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg><arg id=\"2\">%{DATA:arg_2}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA}\"%{HOSTNAME:hostname}:5143\"\]: <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg><arg id=\"1\">%{DATA:arg_1}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA}\"%{HOSTNAME:hostname}:5143\"\]: <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args><arg id=\"0\">%{DATA:arg_0}</arg></args></ios-log-msg>",
            "message", "<%{NUMBER:message_type_id}>%{NUMBER:internal_id}: %{GREEDYDATA}\"%{HOSTNAME:hostname}:5143\"\]: <ios-log-msg><facility>%{WORD:facility}</facility><severity>%{NUMBER:severity}</severity><msg-id>%{WORD:message_type}</msg-id><time>%{SYSLOGTIMESTAMP:message_timestamp}</time><args>%{DATA:args}</args></ios-log-msg>"
        ]
ceskib
  • 140
  • 1
  • 8