Stack Overflow
Stack Overflow

Questions tagged [filebeat]

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis. See more details

1457 questions
4
votes
3 answers

How we can filter namespace in filebeat kubernetes?

I am setting up pipeline to send the kubernetes pods log to elastic cluster. I have installed filebeat as deamonset (stream: stdout) in my cluster and connected output to logstash. Beats is connected with logstash without an issue, now i want logs…
paulpuvi
  • 45
  • 1
  • 1
  • 7
4
votes
3 answers

How to specify pipeline for Filebeat Nginx module?

I have web server (Ubuntu) with Nginx + PHP. It has Filebeat, which sends Nginx logs to Elastic ingestion node directly (no Logstash or anything else). When I just installed it 1st time, I made some customizations to the pipeline, which Filebeat…
Slavik
  • 1,488
  • 1
  • 15
  • 24
4
votes
1 answer

Failed to connect to backoff(async(tcp://ip:5044)): dial tcp ip:5044: i/o timeout

Filebeat is running on Machine B which read logs and push to ELK logstash on Machine A. But in the Machine B filebeat log, it shows the error i/o timeout 2019-08-24T12:13:10.065+0800 ERROR pipeline/output.go:100 Failed to connect to…
LF00
  • 27,015
  • 29
  • 156
  • 295
4
votes
0 answers

Filebeat on Kubernetes modules are not working

I am using this guide to run filebeat on a Kubernetes cluster. https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html#_kubernetes_deploy_manifests filebeat version: 6.6.0 I updated config file with: filebeat.yml:…
RNK
  • 5,582
  • 11
  • 65
  • 133
4
votes
1 answer

How to set kibana index pattern from filebeat?

I am using elk stack with a node application. I am sending logs from host to logstash with filebeat, logsstash formats and send data to elastic and kibana reads from elastic. In kibana i see default index pattern like filebeat-2019.06.16. I want to…
Mitul
  • 108
  • 2
  • 10
4
votes
1 answer

Collect tomcat logs from tomcat docker container to Filebeat docker container

I have a Tomcat docker container and Filebeat docker container both are up and running. My objective: I need to collect tomcat logs from running Tomcat container to Filebeat container. Issue: I have no idea how to get collected log files from Tomcat…
Yachitha Sandaruwan
  • 353
  • 7
  • 19
4
votes
1 answer

Using Kafka as alternative to Filebeats and Logstash

I'm new to the ELK stack and I was just wondering whether if it is possible to ship our log files to Elasticsearch using Kafka. But I need the job of Logstash ( parsing logs using filters like grok ) to be done in Kafka as well. Is this entire…
Dasun Pubudumal
  • 97
  • 1
  • 8
4
votes
1 answer

How to crawl nginx container logs via filebeat?

Problem Statement The NGINX image is configured to send the main NGINX access and error logs to the Docker log collector by default. This is done by linking them to stdout and stderr, which causes all messages from both logs to be stored in the file…
pulkitsinghal
  • 3,855
  • 13
  • 45
  • 84
4
votes
2 answers

Filebeat for gz files

Does filebeat support ingesting from gz files? If I unzip the files before ingesting, they grow in size very big. I did a search on the Elasticsearch forum, but could not find anything useful.
Ijaz Ahmad
  • 11,198
  • 9
  • 53
  • 73
4
votes
1 answer

Sending Filebeat logs to AWS Kinesis

My Problem I have a log pipeline in which logs are written to files and shipped to ElasticSearch using Filebeat. I would like to switch from ElasticSearch to AWS Kinesis, and I wonder what's the right way to configure Filebeat for the new output. My…
Adam Matan
  • 128,757
  • 147
  • 397
  • 562
4
votes
1 answer

How to configure filebeat to handle log rotation?

My settings I have a docker cluster in which supervisord is writing logs to output.log on a shared volume, from which Filebeat is reading and shipping logs to ES. In order to prevent disk-full problems, I have configured supervisord to rotate the…
Adam Matan
  • 128,757
  • 147
  • 397
  • 562
4
votes
1 answer

Is it possible to read compressed files (tar.gz) with FileBeat?

I'm using ELK with FileBeat log shipper and I would like to know if it's possible to read compressed files (like tar.gz) using FileBeats without extracting? Thanks!
asaf
  • 51
  • 1
  • 4
4
votes
2 answers

why do we need filebeat when we can ship logs to Logstatsh

Hi as a newbie to elastic I have a doubt on why we need fileBeat to ship logs to ElasticSearch(ES) or Logstatsh. As far as I knew we can directly read logs from files and send to logstash and from there to ES. If the former is allowed why we need…
k.explorer
  • 291
  • 6
  • 19
4
votes
0 answers

Filebeat - multiline: Ingest XML's without line feed at end of file

I want to ingest XML files to the ELK-Stack. I want one event per XML file. These XML files end without line feed, thus filebeat's multiline codec never forwards the last line of the XML to Logstash. Because of this Logstash's XML filter is then not…
cvanhalt
  • 73
  • 6
4
votes
1 answer

Elasticsearch Filebeat document type deprecated issue

I am currently using ELK 5.5. It appears document_type is now deprecated in Filebeats, but I could not find any example anywhere as to how to implement the same now. This is what I get in my log: WARN DEPRECATED: document_type is deprecated. Use…
user1880957
  • 1,146
  • 3
  • 15
  • 29
1 2 3
…
97 98