Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [logstash]

logstash is a tool for collecting and distributing log events.

logstash is a free and open source tool (apache 2.0 license) for managing events and logs. It can be used to collect and parse logs, and distribution to other indexing systems. It has a web interface for searching and drilling into the logs.

260 questions
0
votes
1 answer

Is there any version of Logstash that can send logs to both Elasticsearch 1.5 and 6.3?

I have an old Elasticsearch v1.5 domain on Amazon AWS, keeping 30 days of logs. It seems that it's not possible to upgrade it, so i have created a new domain with version ES 6.3. Now i have Logstash v2.0 logging to the existing ES v1.5 domain, using…
Christoffer Hammarström
  • 111
  • 7
0
votes
1 answer

logstash won't run as a service

Ive installed logstash v6.3.2 via yum on CentOS7. When I start it via systemctl start logstash it appears to start properly but then exits almost immediately. There are no (apparent) errors shown. (journalctl -f) [2018-07-26T10:32:58,645][INFO…
ethrbunny
  • 2,369
  • 4
  • 41
  • 75
0
votes
1 answer

Need help increasing performance of logstash

We are in the process of deploying an ELK stack and need advice and general recommendations regarding the performance of the cluster and more specifically, logstash. So the current setup we have now is that we have 1 kibana node, 2 logstash nodes…
nillenilsson
  • 153
  • 1
  • 8
0
votes
2 answers

Logstash isn't send info to elasticsearch

I have a Kibana environemtent installed, but isn't working. When I try to configure kibana, kibana says: Couldn't find any Elasticsearch data When I list indexes of elasticseach http://localhost:9200/_cat/indices?v it says: health status index uuid…
David
  • 101
  • 3
0
votes
1 answer

Transform Apache Cookie logs to json with rsyslogd

I am logging cookie headers, some cookies may have due date, some others not, the same is tru for other cookie parameters, I need to implement a centrilaized logging, so before sending those to logstash I want to transform syslog to json. How can I…
Edik Mkoyan
  • 115
  • 5
0
votes
1 answer

Info sent from Logstash via elastic output not showing in Kibana, but file output works fine - what am I doing wrong?

I have an "ELK stack" configuration and, at first, was doing the standard 'filebeat' syslog feeding from logstash with the elasticsearch output plugin. It worked just fine. Now I have added a TCP input port (with assigned "type" for this data so as…
Brendan
  • 73
  • 1
  • 8
0
votes
1 answer

Logstash creating many tcp connections for single hosts

Logstash: 5.3.0 nxlog: nxlog-ce-2.9.1716 Checking nestat I have like 300 established connections even though I have only about 50 hosts. I have like 40 connections for some hosts when I the client's side there is only one active connection. I added…
red888
  • 4,183
  • 18
  • 64
  • 111
0
votes
0 answers

fluentd services die immediately after they start up

Please excuse the naivety in my question, but this is not a subject I know much about at present. My company is currently running kubernetes-managed fluentd processes to push logs to logstash. These fluentd processes start up and fail immediately…
Paul Pritchard
  • 111
  • 5
0
votes
2 answers

Troubleshooting rsyslog integration with ELK stack

I'm trying to configure rsyslog to send logs to logstash on CentOS. So I was following the tutorial. However, after setting up, nothing happens. Everything started ok, not error occurred but no logs in elasticsearch. Here is my…
Sergii Bishyr
  • 101
  • 4
0
votes
1 answer

HAProxy in front of logstash broken pipe

I'm setting up ELK stacks with loadbalancing. Haproxy works fine for ES and Kibana but I'm having issues with Logstash. Here's the haproxy configuration: frontend logstash bind 0.0.0.0:5000 mode tcp option tcpka option tcplog log…
Gab
  • 183
  • 1
  • 8
0
votes
1 answer

How filter already collected info in elasticsearch5.2 on Ubuntu 16.04

Good Evening. I have a ELK stack as follows: Clients with logbeat (windows 2003, 2008 2012, and Linux Ubuntu 16.04) logstash (FreeBSD 11.0) elasticsearch5.2/kibana/nginx 10(Ubuntu 16.04) The problem is that when configuring it I created an index…
Eddy
  • 7
  • 2
  • 10
0
votes
2 answers

ELK vs RabbitMQ for high-volume messaging/document processing?

I've been looking at the ELK stack or RabbitMQ to replace a homegrown system that ingests a large number of files (200-300 million per hour) and operates on then sends them to various locations based on name and content, storing a copy locally.…
Oblivious12
  • 31
  • 2
  • 8
0
votes
1 answer

"mapper_parsing_exception" error from Elasticsearch failing to parse [timestamp] from %{COMBINEDAPACHELOG}

I've configured Logstash to filter httpd_access_log messages and grok the fields associated with COMBINEDAPACHELOG. However, I'm receiving errors like the following: [2017-02-10T15:37:39,361][WARN ][logstash.outputs.elasticsearch] Failed action.…
Justin
  • 3
  • 1
  • 2
0
votes
3 answers

Logstash Grok Parsing Failed

I need to match this entry 2015/10/30 23:58:21 pid 22223 testuser@testserver.example.com 192.168.0.1 [p4/2012.2/LINUX26X86_64/536738] 'test-monitor show' To match this I wrote this Regex P4_DATE…
Prashant Lakhera
  • 713
  • 2
  • 10
  • 25
0
votes
1 answer

Logstash only updates S3 access log indices once per hour

I use a combination of Logstash and the AWS Elasticsearch service to index S3 access logs. The logs are collected in an S3 bucket, processed with the Logstash S3 input filter, renamed after they are processed and then archived in another bucket. I…
Garreth McDaid
  • 3,449
  • 1
  • 27
  • 42
1 2 3
…
17 18