Questions tagged [efk]

121 questions
0
votes
0 answers

How to resolve 'faraday-net_http requires Ruby version >= 2.6.0' error when installing fluent-plugin-elasticsearch?

I'm trying to run docker containers for EFK stack using docker compose. but it keeps throwing me this error : . I tried to install the specified version of faraday-net_http using this command : gem install faraday-net_http -v 2.1.0 and checked my…
0
votes
0 answers

how to map fluentd configuration via configmap using volumes in K8s

Issue here is its copying my entire fluentd-confg.yaml instead of creating fluent.conf file in the fluentd containers. I added the subPath: fluent.conf into volumeMounts and after that there is no any file created inside any of my fluentd…
0
votes
0 answers

ElastAlert Needs to send an alert if EFK stops receiving logs

I want my elastalert to send an email alert if the elasticsearch(efk stack) stops receives logs after a certain time., I tried many ways , but could not able to configure it.. nothing is working for me, can someone please help me to configure…
Ganesh
  • 3
  • 2
0
votes
1 answer

no persistent volumes available for this claim on kubernetes

I'm trying to set up elasticsearch on kubernetes with Helm(helm install elasticsearch elastic/elasticsearch -n efk). I get the error "no persistent volumes available for this claim and no storage class is set". how can i solve Log LAST SEEN …
zoroglur
  • 395
  • 2
  • 18
0
votes
0 answers

How to include the hostname that is added to event logs from fluentbit in fluentd

I Have configured fluent-bit in application pod which is sending nginx access log to fluentd which is deployed as a separate pod and fluentd is responsible for sending logs to elasticsearch but before sending to elasticsearch I want to include the…
Srijan
  • 41
  • 2
0
votes
0 answers

Fluentd generates empty folders

I'm trying to pass an apache_error log to fluentd so it can be transformed into a JSON file. But it only generates file.json as an empty folder and not as a JSON file. Here is my fluentd.conf: @type tail path…
0
votes
0 answers

Best practice for log retention using EFK in AWS EKS

I have an AWS EKS cluster and have built a logging architecture using EFK: Elasticsearch, Fluentd, and Kibana. These are the specific settings. Elasticsearch is deployed using StatefulSet and uses volumeClaimTemplates for volumeMounts in containers.…
SHM
  • 61
  • 1
  • 8
0
votes
0 answers

Fluentbit parse json (Logs)

I'm using fluent-bit to parse logs. Input logs: "log": "2023-02-14T14:11:38,880 INFO [bwEngThread:In-Memory Process Worker-6] c.t.b.p.g.L.C.Log-Event-JSON -…
0
votes
1 answer

ElasticSearch error request params: "wait_for_status=green&timeout=1s"

I am facing error "Readiness probe failed: Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" ) Cluster is not yet ready (request params: "wait_for_status=green&timeout=1s" )" in Elasticsearch…
Rad4
  • 1,936
  • 8
  • 30
  • 50
0
votes
0 answers

EFK with IPv6 does not resolve to geoip data type

I am using td-agent4 in a sidecar. All IPs appear to be v6 and it doesn't resolve to geoip data type. Has anybody successfully used EFK with IPv6? https://github.com/y-ken/fluent-plugin-geoip/issues/97 https://github.com/fluent/fluentd/issues/4004
Kok How Teh
  • 3,298
  • 6
  • 47
  • 85
0
votes
0 answers

FluentD unable to establish connection to ElasticSearch

I am trying to setup an FluentD + ECK on my Kubernetes Cluster. But FluentD is failing to establish connection with ElasticSearch which is on SSL. Error log 2022-10-12 04:55:27 +0000 [info]: adding match in @OUTPUT pattern="**"…
0
votes
0 answers

EFK -> why does it creates so many logs documents - even without any other apps in the cluster?

Running minikube with EFK (elasticsearch, filebeat, kibana). Nothing else is running on the cluster, see here: And all I did was installing it via helm chart: helm install elasticsearch elastic/elasticsearch helm install kibana elastic/kibana helm…
Linux Dev
  • 133
  • 10
0
votes
0 answers

FluentD Can't Read Logs, 'log unreadable. It is excluded and would be examined next time.'

I am trying to deploy an EFK stack on top of an application that stores it's logs in /var/opt/Marklogic/Logs/*.txt (multiple files). I have been able to get the stack running and grabbing the logs from stdout and stderr but when I attach a configmap…
Ilan Rosenbaum
  • 138
  • 1
  • 7
0
votes
1 answer

fluentd(td-agent) output file, Time is missing from the filename

I want to get help. There is a message as below. The message is being received and saved to a file. I want to display the time in the file name, but it's failing. Env td-agent 4.3.1 fluentd 1.14.6 Input message { ... "kubernetes": { "host":…
ddakker
  • 55
  • 7
0
votes
0 answers

EFK stack - fluentd not sending logs to ElasticSearch

I am working on a EFK stack for log management in microservices. Problem statement: Unable to send logs to ElasticSearch from Fluentd, though it was was working earlier with the same setup. Transmission to ElasticSearch has stopped all of…
Mohit Aggarwal
  • 113
  • 3
  • 14
1 2 3
8 9