Stack Overflow
Stack Overflow

Questions tagged [logstash-forwarder]

Logstash Forwarder is a program for securely shipping logs over a network to be ingested by Logstash.

Logstash forwarder is a program for securely shipping logs over a network to be ingested by Logstash.

156 questions
22
votes
6 answers

Docker apps logging with Filebeat and Logstash

I have a set of dockerized applications scattered across multiple servers and trying to setup production-level centralized logging with ELK. I'm ok with the ELK part itself, but I'm a little confused about how to forward the logs to my…
Gianluca
  • 805
  • 1
  • 10
  • 20
8
votes
2 answers

Feeding logstash from azure web app. How?

I have a web app hosted on the azure platform and an ELK stack hosted on a virtual machine also in azure (same subscription) and am struggling to find a way to ship the logs from the app to logstash. A web app stores all its files on a storage…
Sheff
  • 3,474
  • 3
  • 33
  • 35
8
votes
3 answers

Best Docker logging architecture using ELK stack

Recently I am trying to find out best Docker logging mechanism using ELK stack. I am having some questions regarding the best work flow that companies use in production. Our system has typical software stack including Tomcat, PostgreSQL, MongoDB,…
cucucool
  • 3,777
  • 8
  • 48
  • 63
7
votes
0 answers

Pyspark Streaming - How to set up custom logging?

I have a pyspark streaming application that runs on yarn in a Hadoop cluster. The streaming application reads from a Kafka queue every n seconds and makes a REST call. I have a logging service in place to provide an easy way to collect and store…
activelearner
  • 7,055
  • 20
  • 53
  • 94
7
votes
1 answer

Use filebeat or logstash to open gzip files

I might be asking for too much here and I think it's just not implemented yet but I have ELK stack setup similar (almost identical) to how this tutorial does…
Swikrit
  • 352
  • 1
  • 5
  • 21
7
votes
2 answers

Getting IP address of Logstash-forwarder machine

I've setup the Elasticsearch, Logstash, Kibana log viewing tools on my systems. There are 2 machines in my configuration now (Amazon EC2 instances): 54.251.120.171 - Logstash-server where ELK is installed 54.249.59.224 - Logstash-forwarder - sends…
chinmay
  • 1,373
  • 5
  • 15
  • 15
5
votes
4 answers

Tags index with filebeat and logstash

I use logstash-forwarder and logstash and create a dynamic index with tags with this configuration: /etc/logstash/conf.d/10-output.conf output { elasticsearch { hosts => "localhost:9200" manage_template => false index =>…
stecog
  • 2,202
  • 4
  • 30
  • 50
4
votes
3 answers

Read log file from a remote machine with file input plugin using logstash

Presently I have my logs and logstash running on the same machine, so I read my logs placed on my local machine with this config(using pull model) input { file { path => "/home/Desktop/Logstash-Input/**/*_log" start_position…
lambodar
  • 3,495
  • 5
  • 34
  • 58
4
votes
1 answer

JBoss access logs with log rotation

I'm trying to tell my jboss to write an access log with all information I need and use a daily log rotation. So far this is not an issue. The ultimate goal is to send all access log entries to an elk stack using logstash forwarder. Also not that big…
Sebastian Sommerfeld
  • 317
  • 1
  • 5
  • 21
3
votes
1 answer

Filebeat fails to connect to logstash

I'm using two servers on the cloud on one server (A) I installed filebeat and on second server (B) I have installed logstash, elasticsearch, and kibana. So I'm facing problem while sending logs from server A to server B on logstash. My filebeat…
Vinit Jordan
  • 313
  • 1
  • 5
  • 13
3
votes
1 answer

NodeJS Bunyan Logstash Log Indexing

I am using Bunyan and bunyas-lumberjack to send my logs to log stash and index them in elastic search. The problem I am facing is when I am filtering the logs: I am using a basic filter for Logstash : filter { if [type == "json"]{ json { …
alexsc
  • 1,196
  • 1
  • 11
  • 21
3
votes
2 answers

Logstash-forwarder says certificate signed by unknown authority when using a self-signed certificate with SubjectAltName

I'm trying to connect to logstash with logstash-forwarder. Their communication base on ssl so I generate a self-signed certificate follows this. Then I got this error message on logstash-forwarder side: Failed to tls handshake with 9.21.61.19 x509:…
Tyler
  • 43
  • 1
  • 7
3
votes
1 answer

Possible to specify two different codecs in lumberjack?

I have just put up an ELK stack, but I am having trouble regarding the logstash configuration in /etc/logstash/conf.d I have two input sources being forwarded from one linux server, which has a logstash forwarder installed on it with the "files"…
jeffrey
  • 3,196
  • 7
  • 26
  • 44
3
votes
3 answers

Log storage location ELK stack

I am doing centralized logging using logstash. I am using logstash-forwarder on the shipper node and ELK stack on the collector node.I wanted to know the location where the logs are stored in elasticsearch i didn't see any data files created where…
Anand Kadhi
  • 1,790
  • 4
  • 27
  • 40
3
votes
1 answer

Logstash architecture decisions

So we have a bunch of servers running on EC2 Amazon Web Services, and are looking to set up logstash/elasticsearch for distributed logging. From what I read there are several options generally chosen: logstash on each server node, using the File…
MJB
  • 9,352
  • 6
  • 34
  • 49
1
2 3
…
10 11