Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [filebeat]

26 questions
0
votes
1 answer

Mapping fields from a beats log message in graylog

this is a slightly rephrased version of: Whos is eating my fields? (or: how do I get more of the custom fields from my beats message into graylog) i am using filebeat to collect logs from a bunch of docker containers, and then ship them to a…
rmalchow
  • 176
  • 8
0
votes
1 answer

Can't find docker log files for Filebeat

I'm trying to aggregate logs from my Kubernetes cluster into Elasticsearch server. To do that, I've deployed Filebeat on the cluster, but I think it doesn't have a chance to work since in the /var/lib/docker/containers directories, there are no…
Djent
  • 99
  • 2
  • 6
  • 16
0
votes
1 answer

Stop filebeat sending copius metadata

I am sending data from local log files with filebeat to graylog and I am getting a 20x storage overhead compared to the original files. There are a large amount of metadata fields however I can't seem to get rid of them. I have tried many variations…
Damian Games
  • 23
  • 5
0
votes
0 answers

How to have multiple instances of filebeat load balance Netflow input?

I have a very high volume Netflow input stream, and I was hoping that I could run multiple instances of Filebeat and load-balance the Netflow traffic over the Filebeat instances, and then write to a single remote Elasticsearch. I've read about…
Rayne
  • 211
  • 2
  • 14
0
votes
0 answers

Filebeat docker multiline

I'm using filebeat to retrieve log files to Elastic cloud. I'd like to put errors and java exception in one document instead of multiple for each line. I managed to make it work on text log files but it does not seem to work on containers…
Ror
  • 321
  • 3
  • 16
0
votes
1 answer

Can logstash "pull" data?

I have two servers. Server A is running Elasticsearch and Logstash. Server B is running filebeat and is also the server which contains all the logs I'm trying to analyse. Server A is behind a firewall, it can reach out to the internet, but there's…
Aditya K
  • 923
  • 3
  • 13
  • 24
0
votes
1 answer

Filebeat on ECK with AWS Module Fails Due To Metadata Error

We are running an Elastic Stack with ECK in EKS (7.8). We noticed that our filebeat daemonset and the AWS module were not processing logs from S3 and our SQS queues backing up. Looking at the logs on our FileBeat containers, we noticed the following…
JGG Xm8
  • 1
0
votes
1 answer

filebeat log status 30 every sec

I'm learning to use ELK and have a debian PC that runs as a test client. every 30 sec it logs a message : 021-01-18T08:29:59.656-0500#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics":…
Peter
  • 115
  • 1
  • 8
0
votes
1 answer

How to use filebeat to find password in log

I have an Elasticsearch 7.1 and i have configure filebeat to collect all log. I want to check if i have password in log. So anyone have an idea how can i find all password in log using filebeat. Thank you
khaled
  • 21
  • 1
0
votes
1 answer

Grok filter is not working properly

I have Filebeat-7.1 installed in a Debian server, this Filebeat send data from files in this Debian server to server with Logstash 7.6 , here are the files config Filebeat.yml: #=========================== Filebeat inputs…
Ankit
  • 1
0
votes
1 answer

Filebeat kafka input with SASL?

I'm trying to get filebeat to consume messages from kafka using the kafka input. I'm unable to authenticate with SASL for some reason and I'm not sure why that is. The documentation for both Kafka and Filebeat is a little lacking when trying to use…
vane
  • 155
  • 1
  • 2
  • 9
1
2