Logstash filter config is not detected

Question

I use elk (elasticsearch, logstash and kibana) in docker. In logstash I have got input.conf and output.conf. All works fine, but I don't add any grok filters.. If I try add it to input.conf or create new file "filter.conf" but logstash don't see these filters.

My input.conf

input {
    file {
        type => "test"
        path => [
            "/host/var/log/test.log"
            ]
    }
}

My output.conf

output {
    elasticsearch {
        hosts => ["localhost"]
    }
}

My filter:

filter {
  grok {
    type => "test"
    match => [ "%{IP:client}, "%{WORD:method}", "%{URIPATHPARAM:request}", "%{NUMBER:bytes}", "%{NUMBER:duration}" ]

 }
}

Example of log, which is save in test.log: echo 51.0.50.1 POST /index.html 15824 0.049 >> var/log/test.log

What's wrong in this configuration?

"Doesn't see"? You mean you don't end up with the 'client', 'request' (etc) fields, or ?? — Alain Collins, Jan 13 '16 at 16:41

score 0 · Answer 1 · answered Jan 13 '16 at 17:07

Your grok pattern is not well-formed, it should be like below, i.e. with a double quote at the start and at the end and no commas:

filter {
  grok {
    match => { "message" => "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" }
 }
}

Using that filter with your sample log line 51.0.50.1 POST /index.html 15824 0.049, I get the event below which seems correct:

{
       "message" => "51.0.50.1 POST /index.html 15824 0.049",
      "@version" => "1",
    "@timestamp" => "2016-01-13T17:07:15.274Z",
          "host" => "iMac.local",
        "client" => "51.0.50.1",
        "method" => "POST",
       "request" => "/index.html",
         "bytes" => "15824",
      "duration" => "0.049"
}

Logstash filter config is not detected

1 Answers1