logstash stopped match a grok second grok filter

Question

I have two types of logs messages from one source. I am trying to parse them using configuration like this:

filter {
  if [type] == "my_type" {
    grok {
      match => [ "message", "field1:" ]
      break_on_match => false
      add_tag => "field1_message"
    }
  }
  if [type] == "my_type" {
    grok {
      match => [ "message", "field2:" ]
      break_on_match => false
      add_tag => "field2_message"
    }
  }
}

Field1 and Field2 are uniq for each type. My regex and and patterns are correct. When I run this filter, only first part of filter is matched and from second I just receive _grokparsefailure. Can you help me with this?

What do you want to accomplish in the end? Add either of two tags (`field1_message` or `field2_message` in your example) or are your grok expressions actually more complicated? — Magnus Bäck, Oct 16 '14 at 10:28

score 1 · Answer 1 · answered Oct 17 '14 at 03:26

I don't know what is your full config. For me, I have tested with your requirement, it's worked at me. Below is my config:

input {
    file {
        type => "A"
        path => "/path/to/file/A/server.log.1"
    }
    file {
        type => "B"
        path => "/path/to/file/B/server.log.2"
    }

}

filter{
    if [type] == "A" {
            grok {
                    match => [ "message", "field1: %{WORD:field1}" ]
                    break_on_match => false
                    add_tag => "field1_message"
            }
    }
    if [type] == "B" {
            grok {
                    match => [ "message", "field2: %{WORD:field2}" ]
                    break_on_match => false
                    add_tag => "field2_message"
            }
    }
}

output {
    stdout {
            codec => "rubydebug"
    }
}

This is the input for server.log.1

field1: hello

And the corresponding output is

{
   "message" => "field1: hello",
  "@version" => "1",
"@timestamp" => "2014-10-17T03:18:34.421Z",
      "type" => "A",
      "host" => "ABC",
      "path" => "/path/to/file/A/server.log.1",
    "field1" => "hello",
      "tags" => [
    [0] "field1_message"
      ]
}

The second input for server.log.2

field2: world

And the output is

{
   "message" => "field2: world",
  "@version" => "1",
"@timestamp" => "2014-10-17T03:18:33.451Z",
      "type" => "B",
      "host" => "ABC",
      "path" => "/path/to/file/B/server.log.2",
    "field2" => "world",
      "tags" => [
        [0] "field2_message"
    ]
}

Hope this can help you, I am using logstash version 1.4.1.

logstash stopped match a grok second grok filter

1 Answers1