Logstash Processing Multi Line records from file

Question

My log sample is something like this:

2017-01-03 03:38:18 +0000 field1: 123 field2: 321
field3: 1133 field4: 0901
2017-01-03 03:38:19 +0000 field1: 523 field2: 521
field3: 533 field4: 509

Im very new to this. How should I write the regex?

You didn't explictly say, but is the file being *read* locally by logstash, or are you using a shipper like filebeat? — Alain Collins, Jan 23 '17 at 03:19
The file is a local file. It is in the same drive as logstash. — Shawn Sim, Jan 23 '17 at 03:23

score 1 · Accepted Answer · answered Jan 23 '17 at 03:31

1

With the file{} input, you should use the multiline codec (rather than the multiline{} filter), e.g.:

input {
  file {
    path => "..."
    codec => multiline {
      negate => "true"
      pattern => "^%{YEAR}"
      what => "previous"
    }
  }
}

Which you would read as, "if the line doesn't start with a year, keep it with the previous line".

answered Jan 23 '17 at 03:31

Alain Collins

so what does the '^' in the "^%{YEAR}" mean? – Shawn Sim Jan 23 '17 at 03:40
1

The caret means "begins with". Getting the most out of logstash will require a good knowledge of regular expressions. – Alain Collins Jan 23 '17 at 05:14

1 Answers1