2

I am sending logs to logstash using beaver. These logs are multiline in nature so I am using multiline filter but somehow the filter is not able to match the log. There are no errors in the output of the logstash. Here is what I see in logstash:

{"@source":"udp://10.238.161.213:36655/","@tags":[],"@fields":{},"@timestamp":"2013-09-23T16:37:56.653Z","@source_host":"10.238.161.213","@source_path":"/","@message":"{\"@source\": \"file://app1.example.org/ebs/app.log\", \"@source_host\": \"app1.example.org\", \"@message\": \"        This is line3\", \"@tags\": [], \"@fields\": {}, \"@timestamp\": \"2013-09-23T16:38:18.150822Z\", \"@source_path\": \"/ebs/app.log\", \"@type\": \"file\"}","@type":"app"}

Sample log line:

This is a log line
    This is line2
    This is line3

Logstash conf:

input {
  udp {
    type => "app"
    port => 9996
  }
}
filter {
  multiline {
    type => "app"
    pattern => "^\s"
    what => "previous"
  }
}
output {
  elasticsearch { host => "127.0.0.1" }
}

Beaver is running on remote app host and is invoked as:

/usr/bin/python /usr/bin/beaver -D -P /var/lock/beaver.pid -c /etc/beaver/conf -t udp -p /ebs/

and /etc/beaver looks like:

[beaver]
udp_host: logstash.example.org
udp_port: 9996

Any thoughts on why multiine filter is not matching the logs?

Aditya Patawari
  • 1,065
  • 10
  • 23
  • What's the output like? Do you get only the first line? – Abhijeet Rastogi Sep 25 '13 at 07:54
  • I have written the output I see in kibana and elasticsearch: `{"@source":"udp://10.238.161.213:36655/","@tags":[],"@fields":{},"@timestamp":"2013-09-23T16:37:56.653Z","@source_host":"10.238.161.213","@source_path":"/","@message":"{\"@source\": \"file://app1.example.org/ebs/app.log\", \"@source_host\": \"app1.example.org\", \"@message\": \" This is line3\", \"@tags\": [], \"@fields\": {}, \"@timestamp\": \"2013-09-23T16:38:18.150822Z\", \"@source_path\": \"/ebs/app.log\", \"@type\": \"file\"}","@type":"app"}` – Aditya Patawari Sep 25 '13 at 07:56
  • I've tested it against latest version of logstash (1.2.1) and it works perfectly. Can't say about the version you're using. Also, try running with "-v" or "-vv" or "-vvv" to see if you get anything important. Start with something as easy as http://pb.abhijeetr.com/BKUW – Abhijeet Rastogi Sep 25 '13 at 07:58
  • I have same issue, those filter doesn't work on `UDP` but it does work with `file` input – Satish Mar 25 '14 at 02:13

0 Answers0