Logstash File Input stream events from Files
Questions tagged [logstash-file]
221 questions
0
votes
1 answer
Configure logstash to access remote logs
Please can anyone provide me the configuration of logstash to access the log that are located on a remote system. I have tried with IP address, but it says that the plugin failed.
Surendra Deshpande
- 328
- 1
- 3
- 14
0
votes
1 answer
how to get part of the path name and add it to the index
I currently have a file name like this.
[SERIALNUMBER][2014_12_04][00_45_22][141204T014214]AB_DEF.log
i basically want to extract the year from the file (2014) and add it to the index name in logstash conf file.logstash.conf
Below is my conf…
user2181014
- 27
- 2
0
votes
1 answer
How to include regular expression in logstash file input path
I am using logstash to convert tomcat access logs into json format. The access log names are in below format
abcd_access_log.2016-03-15.log
efgh_access_log.2016-02-16.log
The input filter is:
input {
file {
path =>…
PRASANNA SARAF
- 333
- 1
- 4
- 17
0
votes
1 answer
how can i find my generated files logstash?
I'm beginner with ELK stack , so I configured logstash , and when I want to search with ElasticSearch I have no results , so I'm supposed to get a result , because I do my parse on grokdebug and it works very well .
I do my research as…
k. oth
- 33
- 6
0
votes
1 answer
Logstash csv filter create index name based on timestamp
I want to create ES index based on the dates matching from the logfile. I am using logstash CSV filter to process the logs. For instance, the log data appears like below
2016-02-21 00:02:32.238,123.abc.com,data
2016-02-22…
Ganga
- 883
- 12
- 24
0
votes
1 answer
logstash if field exists then grok
I'm trying to create a filter for logstash that will have "general" grok filter for all logs and if some field exists, then I want it to perform a different grok.
The first grok I'm using is
grok {
match => [
"message", "....%{NOTSPACE:name}…
DevopsQueen
- 43
- 1
- 5
0
votes
1 answer
Configure Logstash to wait before parsing a file
I wonder if you can configure logstash in the following way:
Background Info:
Every day I get a xml file pushed to my server, which should be parsed.
To indicate a complete file transfer afterwards I get an empty .ctl (custom file) transfered to…
d.a.d.a
- 1,296
- 1
- 12
- 28
0
votes
1 answer
How can I configure an index pattern?
I have a system using several docker images logging into a syslog-ng.
Syslog-ng is configured to write into files all the streams camming from the other containers.
This part works well and I am getting logs like that:
2016-01-04T20:28:38+03:00…
Rys
- 4,934
- 8
- 21
- 37
0
votes
1 answer
How to write logstash multiline for interlaced log lines from different process threads based on a dynamic identifier
dummy logfile:
[1] test123
[2] test234
[3] test345
[2] test321
[1] test432
[3] test058
[1] test002`
expected result from multiline to merge lines with same id and consider as single event.
[1] test123
[1] test432
[1] test002
Vishal
- 1
- 1
0
votes
1 answer
Cannot listen file in logstash with absolute path
I have logstash installed on path: D:\WORK\ElasticSearch\logstash-2.0.0\bin
and my apps write logs on path: D:\Logs
When I use logstash path to load logs "D:\WORK\ElasticSearch\logstash-2.0.0\bin" with this script i'm able to read it.
input
{
…
Ovidiu Rudi
- 191
- 1
- 15
0
votes
1 answer
Seems Logstash doesn't process the last event/line until the next event is written
I am new to logstash and during my hands on I could see that logstash do not process the last line of the log file.
My log file is simple 10 lines and I have configured filters to process one/two fields and output the json result to a new file.
So…
Veera
- 65
- 1
- 7
0
votes
1 answer
Logstash-ES Data Check
I am currently using logstash-jdbc-plugin to pull data from DB and put it to an index in ES.
How to check if the entire data pulled from DB is getting inserted into Elastic Search index.
The data that is pulled is in millions so can't keep checking…
sri
- 331
- 1
- 4
- 11
0
votes
1 answer
Using FOR statement with Logstash
Can I use the For Statement with Logstash?
And if so, how?
I know we can use the IF statement and I've been using it successfully
hello_its_me
- 743
- 2
- 19
- 52
0
votes
1 answer
Creating a new field using Logstash Filter
I have started writing my own Logstash-filter, based on the example filter provided on Github:
https://github.com/logstash-plugins/logstash-filter-example
My new filter reads from a jar file called Classficiation.jar. I would like to take the…
hello_its_me
- 743
- 2
- 19
- 52
0
votes
1 answer
How to cut and delete elements in an array with Logstash
I have Json logs in an array as follows:
e":[{"n":"3/0/1","st":"CONTENT","v":"Sensortag"},
{"n":"3/0/3","st":"CONTENT","v":"Contiki-develop-20150508-409-g2147b9e"},
{"n":"3/0/13","st":"CONTENT","v":"1970-01-09T21:02:18Z"},
…
arok
- 1
- 1