Stack Overflow
Stack Overflow

Questions tagged [filebeat]

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis. See more details

1457 questions
0
votes
1 answer

Add fields to logstash based off of filebeat data

So, I have a hostname that is being set by filebeat (and I've written a regex that should grab it), but the following isn't adding fields the way that I think it should.. grok{ patterns_dir => "/config/patterns" match =>{ "beat.hostname" =>…
A_Elric
  • 3,508
  • 13
  • 52
  • 85
0
votes
3 answers

Can I load balance logstash cluster with nginx

I have 2 logstash servers that listen on 5000 from multiple filebeats. Only one of the logstash servers is getting traffic. Can I loadbalance these with nginx? Can I do the same with a cluster of elasticsearch servers?
Johnathan Herndon
  • 81
  • 1
  • 5
0
votes
1 answer

What are the advantages of using filebeat as shipper?

Filebeat is used to ship log data into logstash. Logstash read that data at port 5044. input { beats { port => 5044 } } We can directly feed data into logstash. input { file { path => "/tmp/access_log" start_position =>…
learner
  • 4,614
  • 7
  • 54
  • 98
0
votes
1 answer

How can I push static log4j log files which I copy from other systems manually into my LogStash ELK server?

I am manually copying files from 3 server on a daily basis. Because of security reasons I cannot setup an automatic forwarder. So I have 3 directories srvapp1 srvapp2 and dbserver and I copy manually the files into these folders. How can I push…
drgeb
  • 11
  • 1
  • 3
0
votes
1 answer

Inconsistency between Kibana and elasticsearch rest api search output

I've setup ELK Stack. When I query for certain type of log event in kibana it gives me 20 results since start and that is the correct result, but when i query for the same type of log event using rest query either using curl it returns only 10…
Mrunal Gosar
  • 4,595
  • 13
  • 48
  • 71
0
votes
1 answer

logstash and x-forwarded-for on IIS

I just built an ELK server on Windows so I'm new to the process. I've read through the docs but am having trouble parsing out my IIS advanced logs, especially x-forwarded-for data as we're behind a load balancer.. My advanced logging is set up to…
George Nussbaum
  • 3
  • 3
0
votes
1 answer

Centralized ELK vs Centralized EK + Multiple Logstash

We want to set up a common logging interface across all the product teams in our company. We chose ELK for this and i want some advice regarding the set up: One way is to have centralized ELK set up and all teams can use some sort of log forwarder…
mdev
  • 1,366
  • 17
  • 23
0
votes
1 answer

How can i add extra fields in ELK Kibana

I am using ELK with kibana. I am also using filebeat for sending data to Logstash. The i have created look like this { "mappings": { "_default_": { "properties": { "msg":{"type":"string",…
Karl
  • 2,903
  • 5
  • 27
  • 43
0
votes
1 answer

logstash json filter source

I cannot get the message field to decode from my json log line when receiving via filebeat. Here is the line in my logs: {"levelname": "WARNING", "asctime": "2016-07-01 18:06:37", "message": "One or more gateways are offline", "name":…
Dan Schien
  • 1,392
  • 1
  • 17
  • 29
0
votes
1 answer

Running FileBeat on Solaris

I have been trying to run FileBeat on Solaris - SunOS. I was successfully able to run logstash and elasticsearch on it. However, since FileBeat provides a executable to run instead of shell script, is creating the problem. Can any of you please help…
Jigar Sheth
  • 21
  • 7
0
votes
2 answers

How to take input in logstash?

when should I use filebeat , packetbeat or topbeat ? I am new to elk stack. I may sound silly but I am really confused over these. I would appreciate any sort of help.
rresol
  • 323
  • 4
  • 20
0
votes
1 answer

What is default encoding for filebeat to logstash?

I have done such configuration for specific log files for filebeat - files has cp1250 encoding: - document_type: collector encoding: cp1250 ignore_older: 672h log_type: log max_bytes: 134217728 paths: -…
Chameleon
  • 9,722
  • 16
  • 65
  • 127
0
votes
1 answer

Filebeat and bufferring

Sorry, if its a naive question. I've Filebeat is configured to ship data to ES directly. Just incase ES is offline and filebeat harvester found a log to ship, would it buffer, retry and ship? here is what I tried, my docker container generated a…
Sahas
  • 3,046
  • 6
  • 32
  • 53
0
votes
1 answer

Filebeats, get folder name as part of tag

Let's say I have a path something like this: paths: - /var/log/webapp/*.log* - /var/log/webapp/*app*.log* document_type: webapp How would I change this to name the document type based off of the folder name under /var/log/*/…
A_Elric
  • 3,508
  • 13
  • 52
  • 85
0
votes
1 answer

how to make filebeat pick up project-specific configuration file

I have the following yml file, in a project directory not the global file beat configuration directory: filebeat: idle_timeout: 5s prospectors: paths: - "data-log/*" output: elasticsearch: hosts: ["localhost:9200"] Running…
matanster
  • 15,072
  • 19
  • 88
  • 167
1 2 3
…
97 98