I have configured an ELK Server and all there component are on the same servers.. Though, i'm trying to pick my logstash-syslog.conf from the central point that's based on NFS so i dont need to Install logstash on each client..
1) My logstash-syslog.conf file
input {
file {
path => [ "/var/log/messages" ]
type => "test"
}
}
filter {
if [type] == "test" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
if "automount" in [message] {
elasticsearch {
hosts => "oida-elk:9200"
#index => "newmsglog-%{+YYYY.MM.dd}"
index => "%{type}-%{+YYYY.MM.dd}"
document_type => "msg"
}
stdout {}
}
}
2) When I'm running the on the clients to get the data it start the thread and just stucks there ..
[Myclient1 =~] # $ /home/data/logstash-6.0.0/bin/logstash -f /home/data/logstash-6.0.0/conf.d/ --path.data=/tmp/klm
3) after running the above command its shows below logs and then do not proceed anywhere...
[2018-03-05T21:20:51,014][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://oida-elk:9200/"}
[2018-03-05T21:20:51,078][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-03-05T21:20:51,085][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-03-05T21:20:51,101][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//oida-elk:9200"]}
[2018-03-05T21:20:51,297][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125, :thread=>"#<Thread:0x2ea3b180@/home/data/logstash-6.0.0/logstash-core/lib/logstash/pipeline.rb:290 run>"}
[2018-03-05T21:20:51,746][INFO ][logstash.pipeline ] Pipeline started {"pipeline.id"=>"main"}
[2018-03-05T21:20:51,800][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
Please help / suggest anything ..
