Stack Overflow
Stack Overflow

Questions tagged [filebeat]

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis. See more details

1457 questions
5
votes
0 answers

How to handle field collisions from different logging sources in Elasticsearch?

We send logs from a variety of services running in a Kubernetes cluster to Elasticsearch via Filebeat. Some of these services we develop ourselves, others are third-party. We use dynamic mapping in our indices. We've hit an issue where sometimes a…
Matt R
  • 9,892
  • 10
  • 50
  • 83
5
votes
1 answer

The tag "beats_input_codec_plain_applied" present in every document in Kibana

I have set up the ELK stack with the version 7.2.0 : filebeat, logstash, elasticsearch & kibana. When I send my logs to Kibana, I can see a tag "beats_input_codec_plain_applied" in every document. I search through internet but there is no…
d3vpasha
  • 459
  • 7
  • 24
5
votes
1 answer

Filebeat - how control level nested json object parsing - decode_json_fields

how can I control level of decode_json_fields ? max_depth seems not help in my case. goal: parsing '/var/lib/docker/containers//.log' but controlling max json depth (not to generate hundreds of nested fields in elasticsearch index) name:…
AZ-
  • 113
  • 1
  • 10
5
votes
2 answers

Renaming example Kibana dashboards

Is there a way to rename the example Kibana Dashboards that comes with filebeat? The Configuration Dashboard and Configuration Template do help in changing the elasticsearch index name and index name defined in the dashboard. The default kibana…
g_suren
  • 51
  • 1
  • 5
5
votes
1 answer

Filebeat to logstash connection refused

I'm trying to send log files from filebeat->logstash->elastic search. filebeat.yml. But I'm getting the following error in filebeat log: 2017-12-07T16:15:38+05:30 ERR Failed to connect: dial tcp [::1]:5044: connectex: No connection could be made…
Siena
  • 778
  • 10
  • 23
5
votes
1 answer

Debugging Filebeat in the ELK stack

I am having some issues with my ELK system. The client-side work is as follows: Filebeat -> Logstash --> Elastic --> Kibana Parts of our logs don't arrive to Elastic from specific machines. I suspect that the problem is in the log harvesting in…
Green
  • 2,405
  • 3
  • 22
  • 46
5
votes
1 answer

Python - Logging from multiple modules using structlog

I am trying to use Structlog to to log to a file and then using filebeat to send the log to my logging service. I have made the everything work, but I would like to be able to use the same logger across multiple modules, like with Pythons default…
ppoulsen
  • 61
  • 5
5
votes
1 answer

Filebeat multiline kubernetes container logs not working

Hi I have some problem to parse kubernetes containers multi lines using filebeat and logstash. kubernetes log file are located in /var/log/containers/*.log and in a json line structure. Is there something about my configuration is wrong? What did I…
Omri Ziv
  • 51
  • 3
5
votes
2 answers

Filebeat 5.0 output to Kafka multiple topics

I installed Filebeat 5.0 on my app server and have 3 Filebeat prospectors, each of the prospector are pointing to different log paths and output to one kafka topic called myapp_applog and everything works fine. My Filebeat output configuration to…
Gman
  • 2,433
  • 3
  • 26
  • 36
5
votes
1 answer

How to manage input from multiple beats to centralized Logstash

I want to use Elastic Stack for log aggregation for fetching logs from 10 machines. I wish to install Filebeat on 10 machines & grab the logs from each machine and send it to a centralized Logstash server which is installed in a separate machine. In…
Yuvraj Gupta
  • 2,475
  • 16
  • 26
5
votes
1 answer

Filebeat > is it possible to send data to Elasticsearch by means of Filebeat without Logstash

I am a newbie of ELK. I installed first Elasticsearch and Filebeat without Logstash, and I would like to send data from Filebeat to Elasticsearch. After I installed the Filebeat and configured the log files and Elasticsearch host, I started the…
Rui
  • 3,454
  • 6
  • 37
  • 70
5
votes
4 answers

Tags index with filebeat and logstash

I use logstash-forwarder and logstash and create a dynamic index with tags with this configuration: /etc/logstash/conf.d/10-output.conf output { elasticsearch { hosts => "localhost:9200" manage_template => false index =>…
stecog
  • 2,202
  • 4
  • 30
  • 50
4
votes
0 answers

Filebeat kafka input using multiline parser gives no output

Filebeat is configured to use input from kafka and output to file When the multiline setting is turned off, the output is published to a file. But when kafka input is configured with mutiline, no output in the file(file is not even created) Here is…
Sniper
  • 1,428
  • 1
  • 12
  • 28
4
votes
1 answer

Filebeat unable to send data to logstash which results in empty data in elastic & kibana

I am trying to deploy ELK stack in openshift platform (OKD - v3.11) and using filebeat to automatically detect the logs. The kibana dashboard is up, elastic & logstash api's are working fine but the filebeat is not sending the data to logstash since…
Bhavani Prasad
  • 1,079
  • 1
  • 9
  • 26
4
votes
1 answer

FileBeat not sending docker-container logs to Elastic-search

My requirement is to configure filebeat to be able to send logs to elastic search, the source for the file beats is docker container logs. I am using docker to install filebeat, below mentioned are the dockerfile.filebeat.yml & docker-compose files…
k.explorer
  • 291
  • 6
  • 19
1 2
3
…
97 98