I'm trying to make a pipeline that sends xml documents to elasticsearch. Problem is that each document is in its own separate file as a single line without \n in the end.
Any way to tell logstash not to wait for \n but read whole file till EOF and send it?
Can you specify which logstash version you are using, and can you share your configuration?
It may depends on the mode you set: it may be tail or read. it defaults to tail, which means it listens on your file and it waits for default 1 hour before closing it and stopping waiting for new lines.
You may have to change this parameter fro 1 hour to 1 second if you know you have reached the EOF yet:
file {
close_older=> "1 second"
}
Let me know if that works!
