Questions tagged [efk]
121 questions
1
vote
1 answer
Openshift applications logs forward to a single file
how can I forward my applications logs in Openshift to a single file on my filesystem? It is possible with fluentd?
BR
pablo
- 17
- 1
- 4
1
vote
2 answers
fluentd config to parse auth.log
I'm using the following config to have fluentd read the auth.logs and send it to elastic search but i'm faced with an error saying pattern doesn't match and the logs are not pushed to ES.
I'm using the pattern defined in fluentd syslog parser plugin…
nevosial
- 1,034
- 2
- 13
- 20
1
vote
1 answer
How to handle Elasticsearch data when it fills up dedicated volume
I am creating an EFK stack on a k8s cluster. I am using an EFK helm chart described here. This creates two PVC's: one for es-master and one for es-data.
Let's say I allocated 50 Gi for each of these PVC's. When these eventually fill up, my desired…
swagrov
- 1,510
- 3
- 22
- 38
1
vote
1 answer
fluentd : log aggregation from multiple resources using Elastic search FluentD and Kibana
I am having logs coming from various sources and the format of the logs is
[2018-11-20 11:27:41,187] {base_task.py:98} INFO - Subtask: [2018-11-20 11:27:41,186] {child_task.py:355} INFO - Inside poll job status
[2018-11-20 11:27:41,187]…
AmanJ
- 11
- 1
1
vote
0 answers
Kubernetes centralized logging using EFK
I am new to kubernetes and centralized logging. I have installed EFK on my cluster from kubernetes repository and I deleted the flowing code from fluentd-es-ds.yaml file:
nodeSelector:
beta.kubernetes.io/fluentd-ds-ready: "true"
I installed…
Dorin
- 2,167
- 4
- 20
- 32
1
vote
0 answers
EFK. Logs entries doesn't get parsed
We use EFK-stack, where F stands for fluent bit. In my kotlin spring boot application I configured logging as follows with logback and logstash
Roman T
- 1,400
- 5
- 18
- 31
1
vote
3 answers
What is URL for Kibana UI
http://grs-preprodkubemaster01:5601/kibana
I have followed docs and installed Kibana, When I used the service as type: LoadBalancer, the service isn't
coming up, so I deleted the type: LoadBalancer and let it default to ClusterIP, it came up fine.…
sbolla
- 671
- 3
- 22
- 39
1
vote
1 answer
How to proxy app log-level without using the one passed with fluentd?
So i have the following Openshift/Origin architecture installed following the official Openshift/Origin documentation
We also want to use the Aggregated logging setup that comes out of the box, that's why was set by using strictly the Openshift…
Veselin Hristov
- 147
- 2
- 9
1
vote
1 answer
Logging in OpenShift 3.7 issue
I am using Ansible to install logging (EFK) in OpenShift 3.7 with below parameters,…
priyank
- 857
- 2
- 18
- 35
1
vote
1 answer
elasticsearch: max file descriptors [1024] for elasticsearch process is too low, increase to at least [65536]
When I tried to run the logging aggregation I found out the following error generated by elasticsearch:
[2018-02-04T13:44:04,259][INFO ][o.e.b.BootstrapChecks ] [elasticsearch-logging-0] bound or publishing to a non-loopback or non-link-local…
mootez
- 334
- 1
- 7
- 23
1
vote
2 answers
How is K8S_HOST_URL used and what needs it in EFK
Question
What is the purpose of the K8S_HOST_URL configuration parameter in EFK? In EFK, K8S_HOST_URL exists as an environment variable and it looks being used to communicate to the Kubernetes API server by fluentd as being specified in the…
mon
- 18,789
- 22
- 112
- 205
0
votes
0 answers
Log parsing in fluentd
I deploy my Kubernetese projects, I do log management with EFK stack.
This is current log displayed in Kibana. Now I want this log string to be 'broken' into new tags. In this case:
fluentd version: v1.14.6
unmatched_line:
…
zoroglur
- 395
- 2
- 18
0
votes
0 answers
Indexing by traceid and guid[EFK, Elasticsearch, Fluentd]
I print the logs as follows. I want to index for fields such as traceid, guid. How can I do this with a configuration in fluentd and elasticseeach?
Java logger
stringBuilder.append("guid=[").append((guid == null ? "" : guid)).append("]…
zoroglur
- 395
- 2
- 18
0
votes
0 answers
fluentd is sending the same log to elasticsearch twice[EFK]
Fluentd switch is sending two of the same log.
Configuration file below.
I installed on kubernetes with helm chart
I couldn't solve the problem.
How do I edit?
I installed EFK with helm chart.
helm install elasticsearch elastic/elasticsearch…
zoroglur
- 395
- 2
- 18
0
votes
0 answers
EFK Stack on Kubernetes - Collecting logs only from default namespace
I am using EFK stack on Kubernetes and I want FluentD to collect logs only from the applications that are in the default namespace, not the other namespaces.
This is my FluentD config:
apiVersion: v1
kind: ConfigMap
metadata:
name: fluentd-config
…
Daniela
- 1
- 1