Stack Overflow
Stack Overflow

Questions tagged [elastic-beats]

81 questions
0
votes
1 answer

ElasticSearch Ingest Pipeline Null Value Question

If I am using an elasticsearch ingest pipeline in a Filebeat module, and I see the statement of: if: 'ctx.json?.userIdentity?.userName == null' When json.userIdentity.userName is equal to null, what would the key/value pair look like when the if…
Mike R
  • 464
  • 5
  • 16
0
votes
2 answers

Having 1 index vs multiple index in Elastic stack

Lets assume a condition if i have 500k logs in last 15mins which would be better for my elastic stack performance Having 10 index to hold these 500k logs. Having 1 index to hold 500k logs with more shards. Which helps to improve my dashboard…
user17464640
0
votes
0 answers

Shipping a huge amount of logs from an unmanagable folder structure using filebeat

I am trying to ship log files to logstash using filebeat. The folder structure of the input path is currently very branchy and very big. We have a following structure: /////.log The logs…
Luka Klarić
  • 323
  • 2
  • 16
0
votes
0 answers

Can someone explain the detailed technical workflow for communication between elastic agent and fleet server

Need a deeper understanding of how authentication workflow is implemented during the first time installation of elastic agent and how the communication of elastic agent with the fleet server is organized. Which communication protocol is used…
Mann Mehta
  • 1
  • 1
0
votes
0 answers

memory management for elasticsearch

I am trying to calculate good balance of total memory in three node es cluster. If I have three node e.s cluster each with 32G memory, 8 vcpu. Which combination would be more suitable for balancing memory between all the components? I know there…
prat
  • 103
  • 2
  • 10
0
votes
0 answers

Adding Threat Intel fields in Elasticsearch data

How we can add threat intel data for elastic agent logs before they are entering into the Elasticsearch cluster? I am trying to add some new fields to each log before based on the details present in that logs. Example: If you consider…
Pradeep M Nayak
  • 50
  • 2
  • 9
0
votes
0 answers

Possibility of configuring logstash to globally mutate index patterns from a Beats input stream

I have searched on the ELK discussion forum and received no answer. I had this task linked here "Logstash configuration to globally mutate sub index patterns" and it was correctly solved - the logtsash configuration worked and I was able to globally…
es-g
  • 3
  • 2
0
votes
1 answer

logstash - Do not get the values for [log][file][path]

I receive the data from filebeats in logstash but I do not see the value for [log][file][path]. I see othr people get the data in the logstash data, below is what I receive. There is no data from beats for the logpath. Thank you in advance for the…
BKruger
  • 31
  • 4
0
votes
2 answers

Beats installation on AWS-ec2 to send to on-prem ELK

I have to setup jboss over AWS-EC2-Windows server, this will scale-up as well as per the requirements. We are using ELK for infrastructure monitoring for which will be installing beats here which will send the data to on-prem logstash. There we…
Abhishek
  • 3
  • 3
0
votes
1 answer

Multiple beats input ports allowed on Logstash configuration?

I'm trying to achieve getting Logstash to listen to 2 different beats ports - one for filebeat and one for winlogbeat, by setting the conf file to be - input { beats { id => "winlogbeat_plugin" port => 5044 } beats { id =>…
omkar.ghaisas
  • 235
  • 5
  • 19
0
votes
1 answer

How to push complex legacy logs into logstash?

I'd like to use ELK to analyze and visualize our GxP Logs, created by our stoneold LIMS system. At least the system runs on SLES but the whole logging structure is some kind of a mess. I try to give you an impression: Main_Dir | Log Dir |…
Alexander
  • 1
0
votes
3 answers

Can't filebeat use multiple input/output config files in one instance?

Want to deploy filebeat with 3 log definations together. Send to different output targets. --- apiVersion: apps/v1 kind: DaemonSet metadata: name: filebeat labels: k8s-app: filebeat spec: selector: matchLabels: k8s-app: filebeat …
iooi
  • 453
  • 2
  • 10
  • 23
0
votes
1 answer

SSL Setup between Filebeat and Amazon MSK (Kafka)

I have successfully got Filebeat exporting logs to MSK in plaintext mode. When trying to do the same over SSL I’m getting error messages from the Filebeat side Publish failed with circuit breaker is open and SSL Handshake errors in the MSK…
Garreth
  • 1,057
  • 2
  • 9
  • 24
0
votes
0 answers

Connect MySQL through JDBC in filebeats in Elastic Cloud

I am not able to connect MySQL in filebeats in Elastic Cloud. No error I am getting. Can anyone guide me how to do it ? This configuration I'm using input {     jdbc {         jdbc_driver_library => "mysql-connector-java-5.1.46-bin.jar" …
Tech Geek
  • 437
  • 1
  • 4
  • 19
0
votes
0 answers

Is it possible to publish many events to ElasticSearch at once as a transaction? So if one publication fails, ES databse rolls back?

What I'm doing atm is: I gotta fetch a XML file from a server. This XML contains many logs and each log is parsed and turned into an event, a beat.event. What I want is, to send all logs to elasticsearch, or send none. Can I make a huge event and…
Pessoa
  • 1
  • 2
1 2 3
4
5 6