Stack Overflow
Stack Overflow

Questions tagged [filebeat]

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis. See more details

1457 questions
2
votes
1 answer

Elastic Cloud, Filebeat, and Key/Token Authentication

Is it possible configure filebeat to communicate with an Elastic Cloud instance using token authentication? According to the docs, if I'm using a cloud instance I should configure cloud.id and cloud.auth in filebeat.yml cloud.id: "..." cloud.auth:…
Alana Storm
  • 164,128
  • 91
  • 395
  • 599
2
votes
2 answers

How to parse timestamp with underscores in Golang

I'm trying to parse access log timestamp like "2020/11/06_18:17:25_455" in Filebeat according to Golang spec. Here is my test program to verify layout: package main import ( "fmt" "log" "time" ) func main() { eventDateLayout :=…
Alex C
  • 171
  • 1
  • 11
2
votes
2 answers

Filebeat automatically stops without kill

I use filebeat with elk. I started it with nohup command. nohup ./filebeat -e -c filebeat.yml -d "publish" > filebeat.log & Application stopped automatically after one day. close_inactive parameter is not work. Is there any configuration that i…
Turker Ozturk
  • 46
  • 5
2
votes
1 answer

K8s Elasticsearch with filebeat is keeping 'not ready' after rebooting

I'm going through a not very understandable situation. Environment Two dedicated nodes with azure centos 8.2 (2vcpu, 16G ram), not AKS 1 master node, 1 worker node. kubernetes v1.19.3 helm v2.16.12 Helm charts Elastic…
Klaud Yu
  • 23
  • 1
  • 5
2
votes
1 answer

Filebeat to Logstash timezone issue

Using Filebeat, and Logstash 7.9. Customer logfiles have timestamps in local timezone. Want to let Logstash know in which timezone filebeat lives. Am able to do that via adding: processors: - add_locale: format: abbreviation in…
marcel
  • 33
  • 3
2
votes
0 answers

aws-ecs, how to add a filebeat to existing container?

I'm running web service (nginx - uwsgi) on ECS. I'm running the two applications using supervisor. Now I want to add another service (filebeat) which will read logs of the web servers and send to logstash on another machine. I've been told it is…
eugene
  • 39,839
  • 68
  • 255
  • 489
2
votes
1 answer

Transform String into JSON so that it's searchable in Kibana/Elasticsearch

I have Elasticsearch, Filebeat and Kibana running on a Windows machine. Filebeat log has a proper log file and is listening to the path. When I look on the data in Kibana it looks fine. My issue is that the message field is a String. Example of one…
kwoxer
  • 3,734
  • 4
  • 40
  • 70
2
votes
0 answers

Get IIS logs out of AWS elastic beanstalk into Elasticsearch

I want to get my IIS logs off my AWS elastic beanstalk windows web server instances and into Elasticsearch. From what I've read it appears that the easiest thing is to use filebeat for this. I can find plenty of examples of .ebextensions to set up…
dibs487
  • 1,294
  • 4
  • 21
  • 36
2
votes
1 answer

Logstash Error | lumberjack protocol error

We are trying to push our logs from JournalBeat to Logstash. We see some data populated in logstash after the journalbeat service is started. But, not sure why our jounrnalbeat logs are continuously giving the below error…
Vamshi Siddarth
  • 620
  • 1
  • 4
  • 19
2
votes
1 answer

Filebeat not logging to files, always only to syslog

My filebeat (v7.6.0) config has the following: logging.level: debug logging.to_files: true logging.files: path: /var/log/filebeat name: filebeat keepfiles: 7 permissions: 0755 It doesn't create the files, nor does it log to them, it just…
ndtreviv
  • 3,473
  • 1
  • 31
  • 45
2
votes
0 answers

How properly rollover indices in elasticsearch

I harvest logs with filebeat from all docker containers, sending them to logstash and from logstash are forwarded to elasticsearch. I would like to rollover my indices, that are automatically created, if they are too big or too old. There are few…
dorinand
  • 1,397
  • 1
  • 24
  • 49
2
votes
0 answers

Filebeat does not complete on close_eof + --once

Using filebeat 7.5.2: I'm using a filebeat configuration with close_eof enabled and I run filebeat with the flag --once. I can see the harvester reaching eof but the filebeat keeps going. Flebeat conf: filebeat.inputs: - type: log close_eof: true …
José Marques
  • 21
  • 2
2
votes
1 answer

Elasticsearch: Accessing multiple indices created by filebeat

I'm using Elasticsearch 6.8 and i'm not able to access multiple indices in a single query. I've read the documentation and also previous questions, but for some reason i can't figure this out. The setup is fairly standard, i think. I have a filebeat…
srecnig
  • 898
  • 1
  • 7
  • 13
2
votes
1 answer

How to get parts of Filebeat source filename in Logstash

I have an instance of Filebeat (version 7.5.0, running on a Windows Server) monitoring a local folder for log files, and sending this data onwards to Logstash (version 7.5.0, running in a Docker continer). In Logstash I would like to extract one of…
Julian
  • 20,008
  • 17
  • 77
  • 108
2
votes
1 answer

Pull logs from remote server into elasticsearch

The short question is: Is it possible to pull logs (within logfiles) from a remote server and ingest them into the ELK stack. The long story is the following: We have a setup with a DMZ which is publically facing We have an intranet environment…
pfeigl
  • 457
  • 5
  • 12
1 2 3
…
97 98