How to distinct grok filter for similar logs

Question

I have this to kind of logs for dhcpack:

Jun 30 06:34:18 HOSTNAME dhcpd: DHCPACK to IP (MAC) via eth2

Jun 30 06:34:28 HOSTNAME dhcpd: DHCPACK on IP to MAC via eth2

How can I use grok, to use two different matches? I have these two matches for dhcpack, but just use the first:

((%{SYSLOGTIMESTAMP:timestamp})\s*(%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.[for|on] (%{IPV4:dhcp_client_ip})?.[from|to] (%{COMMONMAC:dhcp_client_mac})?.*via (%{USERNAME:interface}))

((%{SYSLOGTIMESTAMP:timestamp})\s*(%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.*[to] (%{IPV4:dhcp_client_ip})?.*via (%{USERNAME:interface}))

Someone can help?

Alain Collins · Answer 1 · 2015-07-08T15:56:20.113

0

I would suggest pulling the common stuff (up to the colon) off first and then processing the more specific stuff with more specific patterns. Some details here.

As shown in the doc, grok{} can take multiple patterns:

filter {
  grok { match => { "message" => [
     "Duration: %{NUMBER:duration}",
     "Speed: %{NUMBER:speed}"
  ] } }
}

By default, it will stop processing after the first match, but that's configurable.

EDIT:

Based on your other comments, you can also branch based on conditionals:

if [myField] == "someValue" {
    grok {
        ...
    }
}
else {
    grok {
        ...
    }
}

In this case, you're running a comparison ("==") or regexp ("=~") to see if you should run a regexp (grok{}). Depending on the full business logic, this seems like a waste.

edited Jul 08 '15 at 15:56

answered Jul 06 '15 at 16:50

Alain Collins

16,268
2
32
55

I want do something like: if (dhcp_action == "DHCPINFORM") { ((%{SYSLOGTIMESTAMP:timestamp})\s*(%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.[for|on] (%{IPV4:dhcp_client_ip})?.[from|to] (%{COMMONMAC:dhcp_client_mac})?.*via (%{USERNAME:interface})) } else { ((%{SYSLOGTIMESTAMP:timestamp})\s*(%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.*[to] (%{IPV4:dhcp_client_ip})?.*via (%{USERNAME:interface})) } It's possible? – Miguel Bessa Jul 08 '15 at 00:38

score 0 · Answer 2 · edited Jan 19 '16 at 06:16

0

I want do something like:

In ((%{SYSLOGTIMESTAMP:timestamp})\s*(%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.[for|on] (%{IPV4:dhcp_client_ip})?.[from|to] (%{COMMONMAC:dhcp_client_mac})?.*via (%{USERNAME:interface}))

get just dhcp_action and use if statement, like:

 if (mCursor != null && mCursor.moveToFirst()) {
         ......
 } else {
         ......
 }

It's possible?

edited Jan 19 '16 at 06:16

Viral Patel

32,418
18
82
110

answered Jul 07 '15 at 13:40

Miguel Bessa

325
2
5
21

1

Yes, you can use conditions, though you may not want to. I've updated my original answer. – Alain Collins Jul 08 '15 at 15:56

score 0 · Accepted Answer · answered Jul 09 '15 at 10:45

I solve the problem with this:

filter { grok { match => ["message", "(dhcpd\S+\s*(%{WORD:dhcp_action_test}))"] } if "DHCPINFORM" in [message] { grok { match => ["message","((%{SYSLOGTIMESTAMP:timestamp})\s* (%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.[from] (%{IPV4:dhcp_client_ip}))"] } } else if "DHCPDISCOVER" in [message] { grok { match => ["message","((%{SYSLOGTIMESTAMP:timestamp})\s(%{HOSTNAME:hostname})\sdhcpd\S+\s(%{WORD:dhcp_action})?.*[from] (%{COMMONMAC:dhcp_client_mac})"] } } else { drop {} }

}

How to distinct grok filter for similar logs

3 Answers3