Stack Overflow
Stack Overflow

Questions tagged [efk]

121 questions
2
votes
1 answer

unable to deploy EFK stack on kubernetes (using kubespray)

i'm trying to deploy EFK stack on production kubernetes cluster (installed using kubespray), we have 3 nodes, 1 master + 2 workers, i need to use elasticsearch as a statefulset and use a local folder in master node to store logs (local storage for…
abdel2020
  • 33
  • 4
2
votes
1 answer

How to enable kibana or elasticsearch alerting or notification

I use EFK (elastic,fluentd,kibana) to analyse my servers logs. I want it to send me notification if a special condition happeend (for example logs are abnormal) I found it is named as watching or alerting in kibana and elastic but as I use basic…
smh
  • 109
  • 1
  • 10
2
votes
0 answers

StatefulSet pods are always pending

I am new to Kubernetes and was following this guide. on deploying EFK stack on a local cluster. After I create the statefulset.yml file, when I try to kubectl create -f statefulset.yml, the pods never startup. On running kubectl rollout status ...…
DarthSett
  • 21
  • 1
  • 5
2
votes
1 answer

Logs are not shown in order after shipped to Elasticsearch using Fluentd

We have an application deployed in Kubernetes and all applications are configured to log to stdout.We use fluentd DaemonSet to collect logs from /var/lib/docker/containers/ folders and ship them to an ElasticSearch cluster. The k8s aggregated log…
Chamila Liyanage
  • 31
  • 4
2
votes
1 answer

Using fluentd, I want to output only one key data from json data

I want to output the kubernetes log to a file. but, I could only output it as json data. I want to output only "message" part to file. How do I choose "message" to print? Which filter should I choose? @type rewrite_tag_filter …
ddakker
  • 55
  • 7
2
votes
3 answers

Readiness and Liveness probes for elasticsearch 6.3.0 on Kubernetes failing

I am trying to setup EFK stack on Kubernetes . The Elasticsearch version being used is 6.3.2. Everything works fine until I place the probes configuration in the deployment YAML file. I am getting error as below. This is causing the pod to be…
karthik ravi
  • 59
  • 1
  • 3
2
votes
1 answer

Monitor the logs of pods from Multiple Kubernetes clusters(Using GCP & Kops) from a common Kibana UI

I have configured a working EFK(Elasticesearch,Fluentd,Kibana) in one of my kubernetes cluster builded in GCP. I have two more clusters and installed the same EFK in remaining too. Now If I want to monitor the logs of each cluster environment,then I…
manu thankachan
  • 433
  • 3
  • 9
  • 19
2
votes
1 answer

[warn]: pattern not match error on fluentd forwarder

I have a fluentd forwarder which gives following error: 2018-02-08 13:07:05 +0000 [warn]: pattern not match: "192.168.6.121 - - [08/Feb/2018:18:37:05 +0530] \"POST /shorten HTTP/1.1\" 200 496 \"-\" \"-\" 6" Following is the content of source block…
drishti ahuja
  • 1,379
  • 2
  • 15
  • 26
2
votes
2 answers

Unable to start fluentD (td-agent)

I have installed fluentD(td-agent) version 2 in CentOs 7. Now I'm unable to start td-agent. While starting td-agent it was throwing some error -- Unit td-agent.service has begun starting up. Feb 02 14:53:39 chninitefk01 fluentd[2282]: …
rolz
  • 591
  • 2
  • 11
  • 23
1
vote
0 answers

Getting "#0 pattern not matched" warning in Fluentd

I used Bitnami helm charts for Fluentd and Official elk helm charts for Elasticsearch and Kibana to deploy EFK stack for log collection in my Kubernetes cluster. But now I'm getting "[warn]: #0 pattern not matched" at the begging (after timestamp)…
M Reza Ghahramani
  • 11
  • 3
1
vote
1 answer

FluentD elasticsearch Plugin @type elasticsearch_data_stream with dynamic datastreams

update-1: I have made some progress on this, by defining a concrete data_stream_name in match. The only thing left is to figure out a way to do dynamic data stream. I am updating the code sample below and marking what I added I have a EFK stack and…
Rishi
  • 313
  • 1
  • 4
  • 18
1
vote
2 answers

Nginx cannot connect fluentd in EFK stack

I am setting up a stack with an application consisting of nginx, redis, mysql, myapp. Nginx proxies requests to myapp. I want to send logs from nginx to EFK stack, but an error occurs when starting the nginx service: Error response from daemon: dial…
Rostislav Udaltsov
  • 79
  • 1
  • 6
1
vote
0 answers

Enable security on kibana after installation with Helm

I installed EFK stack using Helm charts which is working very fine but with zero security. I checked the minimal security for elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html which shows that I…
Anant Agarwal
  • 11
  • 2
1
vote
0 answers

fluentd indices not adding to elasticsearch and kibana

I've deployed EFK stack in IBM Kuberentes cloud by following the step by step guide from this article. Every deployment is done successfully, all EFK stack are deployed fine, but I'm unable to find fluentd indices in elastic search, I'm unable to…
harish hari
  • 93
  • 9
1
vote
1 answer

Fluent Bit Multiline logs issue

I'm trying to implement EFK stack (with Fluent Bit) in my k8s cluster. My log file I would like to parse sometimes is oneline and sometimes multiline: 2022-03-13 13:27:04 [-][-][-][error][craft\db\Connection::open] SQLSTATE[HY000] [2002]…
Murakami
  • 3,474
  • 7
  • 35
  • 89
1
2
3
…
8 9