Stack Overflow
Stack Overflow

Questions tagged [elastic-beats]

81 questions
1
vote
1 answer

If then else not working in FileBeat processor

I'm trying to setup some processors in a filebeat.yml to process some logs before sending to ELK. An important part of the processing is determining the "level" of the event, which is not always included in the line in the log file. This is the idea…
ziv
  • 37
  • 3
  • 9
1
vote
0 answers

Shipping logs from network share using Filebeat on Windows

The problem statement: I have an application running on Windows. I want to ship logs files from this application to ELK fronted by Kafka. The challenge: This application writes a lot of process metadata to disk under a directory location. This…
Prabodh Mhalgi
  • 805
  • 9
  • 25
1
vote
3 answers

How to ingest a .json file from persistent volume to elasticSearch

I have a web crawler that crawls thousands of websites and store that in a persistent volume on Kubernetes. After that pod is terminated, I want to ingest the .json file inside the PV into ECK. I have achieved running elastic search and Kibana…
Jad Sakr
  • 11
  • 1
1
vote
1 answer

ELK MetricBeat - Monitor remote mysqlDB

I have installed ELK stack (With metricbeat) on ServerA and want to monitor mysql on ServerB. I added db host details on ServerA mysql.yml metribeat module file (/etc/metricbeat/modules.d/mysql.yml) - module: mysql metricsets: - status …
ashwin
  • 332
  • 2
  • 16
1
vote
2 answers

How to parse a mixed custom log using filebeat and processors

I'm trying to parse a custom log using only filebeat and processors. I wouldn't like to use Logstash and pipelines. Below a sample of the log: TID: [-1234] [] [2021-08-25 16:25:52,021] INFO…
Kleyson Rios
  • 2,597
  • 5
  • 40
  • 65
1
vote
0 answers

call jinja2 template within Ansible role at Playbook level

I have a folder structure as Ansible: - roles - elastic.beats - templates filebeat-inputs.yml.j2 From my playbook, I am calling it as below: - { role: elastic.beats, beat: "filebeat", beat_conf:…
Buggy B
  • 623
  • 7
  • 18
1
vote
1 answer

Getting "could not be deployed. Enable it" while deploying Functionbeat as AWS Lambda with SQS trigger

I am trying to deploy Functionbeat to AWS Lambda from a Windows machine, I have followed the documentation I could find What I did: 1- Downloaded Functionbeat from https://www.elastic.co/downloads/beats/functionbeat (Windows zip-64) 2- Configure AWS…
Yahya Hussein
  • 8,767
  • 15
  • 58
  • 114
1
vote
1 answer

Mock an ELK Beat output to Logstash with Postman

I have Elastic Beats (like metricbeat and auditbeat) that send data to logstash. For testing purposes, I'd like to mock some data from the beats to the logstash input. I'm using the logstash output plugin in the beats (yaml config…
Mathieu Rollet
  • 2,016
  • 2
  • 18
  • 31
1
vote
0 answers

Filebeats saying dropping the line but still see in Kibana

I'm working on the ELK stack. I'm trying to push the wso2 application logs to Elastic Search. I configured Filebeats to read the lines that include DCS. I see the filebeats log in the terminal as dropping the line doesn't match the pattern…
saitama
  • 699
  • 1
  • 9
  • 21
1
vote
0 answers

How to handle overlapping logs with filebeat?

is there a way to handle overlapping logs with filebeat? here's a sample of what i have: [17/02/2020 07:53:27:748] 00000 I >> message1 (start of event) [17/02/2020 07:53:27:751] 00000 I @ message2 [17/02/2020 07:53:27:785] 00004 I >> message1 (start…
Amine Maalfi
  • 145
  • 9
1
vote
0 answers

Fielddata is disabled on text fields by default kibana

Fielddata is disabled on text fields by default Hello, Something weird happend lately in my elasticsearch setup. Metricbeat index got replaced with a new name like the old one is like metricbeat.x.date, but now the index is showing as metricbeat.…
sai kumar
  • 15
  • 2
  • 7
1
vote
1 answer

How to setup filebeat monitoring via stack monitoring in kibana?

I'm trying to setup monitoring for filebeat via kibana stack monitoring ui. But when I tried to do this, I'm unsble to see the filebeat stats on this page, I can just see ELK stack stats. This's my filebeat.yml config filebeat.inputs: - type: log …
megamind79
  • 69
  • 3
  • 11
1
vote
3 answers

Why use Beats if i can post directly to Elasticsearch?

Recently i have been reading into Elastic stack and finding out about this thing called Beats, which basically used for lightweight shippers. So the question is, if my service can directly hit to Elasticsearch, do i actually need beats for it?…
DemiDust
  • 313
  • 1
  • 3
  • 19
1
vote
1 answer

How do Beats receive configuration changes from Beats Central Management (i.e REST)?

I am trying to find how the Beats Central Management console in Kibana deploys configuration changes to enrolled Beats. I have searched the documentation of both Beats and Beats Central Management, but have seen no clear detail specifying how these…
Maria
  • 121
  • 1
  • 8
1
vote
0 answers

How to limit logging level via processors?

I use journalbeat to ship journald events to logstash (and then ultimately to elasticsearch). I am interested only in higher priority messages. At the very top of my journalbeat.yml file I have processors: - drop_event: when: range: …
WoJ
  • 27,165
  • 48
  • 180
  • 345
1
2
3 4 5 6