Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [logstash]

logstash is a tool for collecting and distributing log events.

logstash is a free and open source tool (apache 2.0 license) for managing events and logs. It can be used to collect and parse logs, and distribution to other indexing systems. It has a web interface for searching and drilling into the logs.

260 questions
3
votes
2 answers

How to send ip address to logstash using input as "file"?

I am sending few logs to logstash central server using another logstash on client as shipper. The input type is "file". The messages are received fine on server but it does not reflect the IP address of the client. It sends the hostname instead in…
Aditya Patawari
  • 1,065
  • 10
  • 23
2
votes
0 answers

Logstash to aggregate logs into ElasticSearch

I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS. filebeat is already running in EKS to aggregate Kubernetes container logs. I have configured rsyslog client with…
roy
  • 119
  • 1
  • 2
  • 15
2
votes
0 answers

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", "connection": "keep-alive", …
vkpro
  • 21
  • 1
  • 2
2
votes
1 answer

Filebeat can't connect to logstash on another server

Filebeat (11.11.11.11) can't connect to logstash (22.22.22.22) on another server (connection reset by peer). But filebeat services from other servers can do it. Also I can connect from this server(11.11.11.11) using telnet to this port (telnet…
Dmitry
  • 179
  • 1
  • 2
  • 8
2
votes
1 answer

ELK logstash and core grok patterns

I'm evaluating the ELK stack with filebeat & logstash across a diverse range of applications/ servers. I understand the power of customising my own grok patterns for each application/log, but to get running initially it seems very inefficient to…
Dan Poltawski
  • 141
  • 1
  • 3
2
votes
0 answers

IIS not UTF-8 encoding logs correctly. Bug?

My server's log file is set to encode in UTF-8, it does not seem to be encoding the text correctly. For example the cookie section sometimes has a £ sign in. But in the log it is written as 0xA3 not 0xC2 0xA3. Is this a bug? I'm trying to pass the…
Mesh
  • 213
  • 3
  • 10
2
votes
0 answers

Logstash cannot find logstash.yml although it exists

Testing logstash as instructed here. This is the command I execute (it doesn't work without the --path.settings too): /usr/share/logstash/bin/logstash -f logstash-simple.conf --path.settings /etc/logstash/logstash.yml But I am getting this…
kev
  • 261
  • 1
  • 5
  • 13
2
votes
1 answer

Trouble bringing in CloudWatch data to Logstash

Just getting started with a new ELK setup (never used it before, just trying to learn it). I have Logstash 2.2.4 running on ubuntu 14.04 LTS. After putting a yaml file down with my monitor user's AWS credentials (policy configured as per the…
Paul
  • 998
  • 1
  • 11
  • 19
2
votes
1 answer

How do I feed Elasticsearch my Snort alert log?

I started off yesterday with an ELK howto and got ELK up and running rather easily. Next thing I wanted to do was plug my Snort alert log into it. I configured Logstash (shown below) with a filter and an absolutely nasty Grok regex to split up all…
Server Fault
  • 3,714
  • 12
  • 54
  • 89
2
votes
0 answers

Logstash nmap codec @timestamp match

I am using Logstash to input NMAP scan data into Elasticsearch. input{ http{ host => "127.0.0.1" port => 8000 codec => nmap } } filter{ date{ match => […
user79067
  • 33
  • 3
2
votes
1 answer

How to format log data before forwarding them as JSON to elasticsearch?

I log all events on a system to a JSON file via syslog-ng: destination d_json { file("/var/log/all_syslog_in_json.log" perm(0666) template("{\"@timestamp\": \"$ISODATE\", \"facility\": \"$FACILITY\", \"priority\": \"$PRIORITY\", \"level\":…
WoJ
  • 3,607
  • 9
  • 49
  • 79
2
votes
1 answer

Running logstash under SELinux

I want to run logstash as root to allow it to read all logs (granting it access to every log is very tiresome). However, I don't want it running amok on my server, I thought about constraining it under SELinux. The options I see are: Create an…
Nitz
  • 1,038
  • 1
  • 8
  • 18
2
votes
0 answers

How do I update Logstash data to Redshift real-time?

I am looking for a way to build a pipeline for storing the logs data (I use the ELK stack) to AWS Redshift, for analytics. I have gone through the Output plugins for ElasticSearch, and found one for writing data to AWS S3. But, it writes data in…
Dawny33
  • 143
  • 1
  • 7
2
votes
2 answers

Grok Issue with Multiple IP's in NginX Logstash

i've got an issue with logging from my webservers, which has an elb and then a varnish layer in front of nginx layer. varnish is setup properly for X-Forwarded-For and logs come through normally with the correct 'client.ip' being logged. however,…
geniestacks
  • 65
  • 1
  • 2
  • 7
2
votes
2 answers

Pipe directly to Logstash?

I would like my log data for servers such as Nginx to go directly to Logstash over a network. Logstash has TCP and UDP socket handlers built-in that would be perfect for receiving the data, but sending it is a problem. I know that there is the…
Brad
  • 1,419
  • 22
  • 43
1 2 3
…
17 18