Stack Overflow
Stack Overflow

Questions tagged [filebeat]

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis. See more details

1457 questions
-1
votes
1 answer

{"type":"mapper_parsing_exception","reason":"failed to parse field [user_agent.version]

i m sending Nginx logs via filebeat -> elastic search -> Kibana . But already have issue with some logs . It s look like this type of log is parsing without any problem : 66.249.76.123 - - [24/Apr/2020:17:24:51 +0200] "GET / HTTP/1.1" 200 5249 "-"…
Samuel Stanislav
  • 31
  • 1
  • 3
-1
votes
1 answer

filebeat send messages to certain index

I have an installed pair elasticsearch - logstash - kibana, 2 clients: ELKclient1 and ELKclient2. Filebeat is installed on clients. I need that both clients write logs in separate index, ELKclient1 in index test-%{+YYYY.MM.dd, ELKclient2 in index…
aleksss
  • 11
  • 1
-1
votes
1 answer

filebeat/logstash convert json to csv format

Need an advice and sample code as i am new to filebeat/logstash configuration. The harvester sends a dsv file to filebeat and the filebeat is taking the dsv input and sending an json output to logstash server. Can I send the dsv file format to…
krsna
  • 1
-1
votes
1 answer

Does filebeat add metadata while sending logs to elasticsearch?

I'm trying to send systemd logs to elasticsearch via filebeat. I see the following metadata field in elasticsearch for one of the log entries. "meta" : { "cloud" : { "instance_id" : "xxxxxx", "machine_type" : "xxxxxx", …
Paridhi
  • 25
  • 6
-1
votes
1 answer

Can i avoid repetition in filebeat input settings?

I have an input settings like this (Proof Of Concept) and i will add more prospectors further on. Can i avoid repetition of the multiline properties? filebeat.prospectors: - type: log enabled: true paths: -…
oygen
  • 467
  • 1
  • 7
  • 13
-1
votes
1 answer

I have a separate machine for elasticsearch. It is 500GB, but logs are consuming full memory in 24 hours. How do I compress it and free memory?

[2019-08-01T13:20:48,015][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({“type”=>“cluster_block_exception”, “reason”=>“index [metricbea...delete (api)];“})
karthik sl17
  • 19
  • 1
  • 7
-1
votes
2 answers

Failed to restart heartbeat.service Unit not found

Jul 05 14:00:25 ip-19-0-1-16.ec2.internal heartbeat[13996]: 2019-07-05T14:00:25.518Z ERROR instance/beat.go:877 Exiting: could not create monitor: job err missing required field accessing ‘heartbeat...heartbeat.yml’) Jul 05 14:00:25…
karthik sl17
  • 19
  • 1
  • 7
-1
votes
4 answers

ELK Community Beats to gets AWS logs on ELK

I have my ELK server setup on Ubuntu 16.xx (AWS EC2 instance). 1 - I need to monitor my AWS services (SNS, SQS, SES, Lambda, Ec2, S3 ELB CloudWatch .. etc.) 2 - how can I get the real-time logs from my aws service and post to my Kibana dashboard…
shahid
  • 11
  • 6
-1
votes
1 answer

How to process and label the log data before showing in Kibana, using filebeat to logstash forwarding

I'm trying to learn ELK . I have setup Filebeat on one host which is forwarding the logs to logstash on some other server which is forwarding logs to elasticsearch. The logs being forwarded by filebeat are /var/log/messages and…
Learner
  • 1,544
  • 8
  • 29
  • 55
-1
votes
1 answer

Data exchanged between two linux servers

I am Using Filebeat and Kafka for Log management. I have a query that sends from data Filebeat to Kafka. How to know if the data is in plaintext format or is it encrypted? Please help me. Thank you.
user3013193
  • 1
-1
votes
2 answers

Filebeat command not found

I am running ELK in a docker container at localhost, I am trying to start filebeat (not through docker container) but I am receiving the error below. /etc/init.d/filebeat: command not found Has anyone had similar issues or any idea how to resolve…
many
  • 67
  • 2
  • 9
-1
votes
1 answer

How to install logstash-forwarder for multiple logstash server?

Currently we are working on forwarding logs to 2 different logstash servers. We cannot figure out a way with which we can install logstash-forwarder on a single machine. Is it possible with logstash-forwarder forwarding logs to multiple logstash ??…
Grin like a Cheshire cat
  • 397
  • 2
  • 11
  • 25
-1
votes
1 answer

Logstash is not working and consume 99% CPU

We have the following infra structure to index application log data to ELK. filebeat -------> Logstash ------> Elastic search-----> kibana All were working fine but suddenly Logstash server consume 99.9% CPU after which no indexing is happening. In…
Sarathy
  • 442
  • 2
  • 9
  • 20
-2
votes
1 answer

How to read stdout stderr logs in kubernetes within pod using filebeat or logstash,fluentd

need suggestions how can i capture containers log using stdout or stderr ? within a pod on following use case ? my pod contains 3 containers where i want third container to capture logs by using any of these longing options filebeat, logstash or…
anil
  • 1
  • 1
-2
votes
1 answer

How can I pick logs from a specific directory and display on Kibana

I am a newbie on elastic-stack. I setup a elastic-stack and filebeat on Ubuntu 16.04 on local environment. Now I want to read log files from a specific directory. In my case LogFile is my directory which is placed on Ubuntu desktop. I want to know…
Muhammad Hassan
  • 235
  • 1
  • 2
  • 7
1 2 3
…
97
98