Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [logstash]

logstash is a tool for collecting and distributing log events.

logstash is a free and open source tool (apache 2.0 license) for managing events and logs. It can be used to collect and parse logs, and distribution to other indexing systems. It has a web interface for searching and drilling into the logs.

260 questions
0
votes
1 answer

ElasticSearch Delayed Indexing

I currently have the following setup: syslog-ng servers --> Logstash --> ElasticSearch The syslog-ng servers are load balanced and write to a SAN location where Logstash just tails the files and sends them to ES. I'm currently receiving around 1,300…
Eric
  • 1,383
  • 3
  • 17
  • 34
0
votes
2 answers

Logstash filtering syslog by host group?

I've got an Elasticsearch/Logstash/Kibana instance running, which I'm merrily stuffing with syslogs from a variety of hosts. Having built it to scale - with multiple logstash syslogd listeners, and multiple ES nodes - it's doing quite nicely for…
Sobrique
  • 3,747
  • 2
  • 15
  • 36
0
votes
1 answer

ELK: LogStash to read log files from remote Samba-mapped network drives

I'm new to ELK, and I would like to set up a solution to index Microsoft IIS and applicative .NET logs with ES. I'm aware about different approaches: 1) [app servers: log files ➔ Logstash] ➔ [collecting server: Redis ➔ Logstash] ➔ [ES cluster: ES ➔…
Maxim Gueivandov
  • 83
  • 1
  • 9
0
votes
1 answer

Elasticsearch Cluster Configuration

Brand new to Elasticsearch. I've been doing tons of reading, but I am hoping that the experts on SO might be able to weigh in on my cluster configuration to see if there is something that I am missing. Currently I am using ES (1.7.3) to index some…
NationalDonut
  • 3
  • 2
0
votes
1 answer

Kibana: Timestamp field looks different from the Table and JSON fields in the UI

From the combined screenshot can you tell why there is a difference in timestamp for a log entry in different views in Kibana UI. Update: My timezone is +5:30 GMT, but I need the logs to show up based on the time in the logfile. The server which…
Ajov Crowe
  • 133
  • 1
  • 5
  • 12
0
votes
2 answers

Logstash: UNIX Epoch time not getting converted to readable format

I have setup an ELK stack and I am trying to parse squid log entries. And I am having a problem trying to convert the following UNIX/Epoc time as 1442469455.757 to a human readable format. While trouble shooting I get the following error: Received…
Ajov Crowe
  • 133
  • 1
  • 5
  • 12
0
votes
1 answer

Elasticsearch index created with fieldtype mapping integer, recognized as "string" in Kibana 4

I have created an index in elasticsearch 1.7 like this: curl -XPUT 'http://localhost:9200/test' -d ' { "test" : { "properties" : { "user" : {"type" : "string", "index" : "not_analyzed"}, "message" : {"type" :…
scaarup
  • 3
  • 3
0
votes
1 answer

Logstash output to AlienVault (OSSIM)

I have the following topology: rsyslog clients -> logstash server -> elasticsearch & another logstash & alienvault & local file The problem is that alienvault wants just the raw message, without any json fields. How could i send only to alienvault…
Constantin Manea
  • 1
  • 3
0
votes
1 answer

How to select only specific log from records with imfile module?

We have a Logstash agent that writes its internal logs to a file in fixed format (a message written in a single row, here it's formatetd for readability): { :timestamp => "2015-08-20T18:24:07.458000+0300", :message => "SIGINT received.…
Aliaxander
  • 101
0
votes
0 answers

How to extend a field with a multiline message

I have a logstash conf like so: input { lumberjack { port => 5000 type => "logs" ssl_certificate => "/path/to/mycert" ssl_key => "/path/to/mykey" } } filter { if [type] == "logs" { if [message] =~ /^\d\d\d\d/ { grok…
goofdad
  • 101
  • 2
0
votes
3 answers

Getting cloudfront logs into logstash: ERROR: is not a legal argument to this wrapper, cause it doesn't respond to "read"

Logstash version 1.5.0.1 I am trying to use the logstash s3 input plugin to download cloudfront logs and the cloudfront codec plugin to filter the stream. I installed the cloudfront codec with bin/plugin install logstash-codec-cloudfront. I am…
Peter M
  • 973
  • 2
  • 15
  • 27
0
votes
2 answers

ELK stack: Can't load Kibana 4

I have installed Elasticsearch, Logstash, Kibana and Nginx following this guide, exept I used the latest version of everything. https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04 The…
mfernholm
  • 3
  • 1
  • 3
0
votes
1 answer

Logstash is not processing MySQL logs correctly

As you can see, the message is not just the message, but also contains the date and timestamp. . . This is the MySQL Log: /var/log/mysql/error.log 150630 9:01:29 [Warning] Access denied for user 'test1'@'localhost' (using password: YES) 150630 …
ujjain
  • 3,983
  • 16
  • 53
  • 91
0
votes
0 answers

logstash doesn't create index in elasticsearch

I followed this tutorial from Digital Ocean on how to install an ELK stack on a CentOS 7 machine. Digital Ocean ELK Setup CentOS It seemed pretty good, and got me as far as having an initial Elastic Search node working correctly and have kibana 4…
user99201
  • 287
  • 2
  • 8
  • 22
0
votes
0 answers

Unable to fetch mapping. Do you have indices matching the pattern?

I am new to ELK and recently installed ELK on Windows Server 2008 and on Windows 7 machine. But on both the places if I open Kibana I get the message: "Unable to fetch mapping. Do you have indices matching the pattern?" My logstash confiuration file…
user3560812
  • 1
  • 1
1 2 3
…
17 18