Stack Overflow
Stack Overflow

Questions tagged [filebeat]

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis. See more details

1457 questions
3
votes
2 answers

How to use variables in filebeat.yml file

I am using env variables in filebeat.yml, it is failing to parse the variables. filebeat.yml output.elasticsearch: hosts: [$ELASTICSEARCH_HOST] template: name: "filebeat" path: "fields.yml" overwrite: false protocol:…
BSM
  • 173
  • 3
  • 13
3
votes
1 answer

Importing PCAP into Elasticsearch

I'm trying out Elasticsearch for the very first time. I've downloaded Elasticsearch and Kibana and everything seems to run fine. I can visit http://localhost:5601 and view Kibana without errors. I've made some traces with wireshark/tshark and…
Alfred Balle
  • 1,135
  • 4
  • 16
  • 32
3
votes
1 answer

Filebeat fails to connect to logstash

I'm using two servers on the cloud on one server (A) I installed filebeat and on second server (B) I have installed logstash, elasticsearch, and kibana. So I'm facing problem while sending logs from server A to server B on logstash. My filebeat…
Vinit Jordan
  • 313
  • 1
  • 5
  • 13
3
votes
1 answer

Where are logs of docker nginx conainter stored in host

I use default nginx image and Filebeat to read logs and send them to ELK. Both containers (nginx container and Filebeat container) are on the same host machone. Here is Dockerfile for nginx image FROM nginx COPY . /usr/share/nginx/html/ EXPOSE…
Nurzhan Aitbayev
  • 797
  • 1
  • 10
  • 23
3
votes
3 answers

Cannot run logstsh on windows

I am trying to get the logs from logstash and send it to elasticsearch for visualising the logs using kibana but I am getting an error while running this code from logstash\bin directory logstash -f logstashpipline.conf The error says Error:…
glaltv
  • 61
  • 1
  • 3
3
votes
1 answer

ELK apache spark application log

How to configure Filebeats to read apache spark application log. The logs generated is moved to history server, in non readable format as soon as the application is completed. What is the ideal way here.
Thelight
  • 359
  • 1
  • 5
  • 15
3
votes
2 answers

Filebeat is not creating index in Elasticsearch

I'm setting up Filebeat to send logs to Elasticsearch. This is my filebeat.yml: filebeat.prospectors: - type: log paths: - '/var/log/project/*.log' json.message_key: message output.elasticsearch: hosts: ["localhost:9200"] I have this file…
Héctor
  • 24,444
  • 35
  • 132
  • 243
3
votes
0 answers

Support multiple multiline patterns in ELK log parsing

TL;DR: how can I best support multiple multiline patterns in ELK log parsing? With the popularity of Docker, Python and ELK, one would think that getting properly parsed Python logs out of docker into ELK would be incredibly simple, but I can’t find…
AgileZebra
  • 593
  • 6
  • 14
3
votes
1 answer

What is the architecture for recover logs from a storage account to a Elasticsearch deployed on KUBERNETES?

I am working on the evolution of a siem soc, and my actual issue is to recover my logs from my storage account on Azure to a Elasticsearch-data deployed on a pod on KUBERNETES. So I would like to know the the right approach for that. With filebeat…
Diamant Eméraude
  • 31
  • 2
3
votes
2 answers

Multi-line logs into ES from filebeat deployed as Kubernetes Daemonset

I have setup filebeat as a daemonset in kubernetes to forward logs to ES + kibana from docker containers. (by referencing https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html) And the logs are forward sucessfuly. The…
Dushmantha
  • 2,911
  • 1
  • 14
  • 21
3
votes
1 answer

Filebeat To Logstash -InvalidFrameProtocolException

I am trying to load data from filebeat into logstash. While loading , while running the command-> bin/logstash -f first-pipeline.conf --config.reload.automatic , following error is encountered: [2018-06-05T11:30:43,987][INFO…
Umang Bhargava
  • 43
  • 2
  • 6
3
votes
3 answers

How do I disable filebeat's close_inactive setting?

My filebeat (container from docker.elastic.co/beats/filebeat:6.1.2) harvesters are being closed by the close_inactive and I don't want them to be. The documentation from close_inactive from here states When this option is enabled, Filebeat closes…
Dave Lugg
  • 2,326
  • 2
  • 15
  • 23
3
votes
2 answers

filebeat-index-template.json for ElasticSearch 6.2.4

I am running ElasticSearch 6.2.4. I tried to create Filebeat index template, but got the following error { "error" : { "root_cause" : [ { "type" : "mapper_parsing_exception", "reason" : "No handler for type [string]…
invisal
  • 11,075
  • 4
  • 33
  • 54
3
votes
1 answer

Grok doesn't match multiline log entries properly?

I've been going at this for weeks now and I can't seem to wrap my head around what's wrong about this. I'm trying to get all of these lines to fit into a multiline match with grok, but it only picks up the last one, and even discards the digit at…
Atombob
  • 114
  • 1
  • 9
3
votes
1 answer

Filebeat.prospectors: input_type vs type

I've got a couple of filebeat.yml files on different servers. One has: filebeat.prospectors: - input_type: log and the other has: filebeat.prospectors: - type: log Are input_type and type synonymous?
Snowcrash
  • 80,579
  • 89
  • 266
  • 376
1 2 3
…
97 98