Seems Logstash doesn't process the last event/line until the next event is written

Question

I am new to logstash and during my hands on I could see that logstash do not process the last line of the log file.

My log file is simple 10 lines and I have configured filters to process one/two fields and output the json result to a new file.

So when logstash is running I open the monitored file and add one line to the end of file and save it. Nothing happens. Now I add one more line and the previous event shows up in the output file, and similarly for the next events.

How to resolve this behavior ? Is something wrong with my usecase/config ?

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
    file {
        path => "C:\testing_temp\logstash-test01.log"
        start_position => beginning 
    }
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
    grok {
        match => { "message" => "%{IP:clientip} pssc=%{NUMBER:response} cqhm=%{WORD:HTTPRequest}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
    file {
    path => "C:\testing_temp\output.txt"
}
}

score 0 · Accepted Answer · answered Nov 13 '15 at 13:20

please make sure to add a a newline at the end of your line when manually inserting. Logstash will pick up your changes as soon as it detects that the line is "finished".

Your usecase is ok. If you add:

stdout { codec => rubydebug }

To your output section you will see the events immediately in your console (nice for debugging/testing).

Seems Logstash doesn't process the last event/line until the next event is written

1 Answers1