Fluentd is open-source and distributed data collector, which receives logs in JSON format, buffers them, and sends them to other systems like Amazon S3, MongoDB, Hadoop, Loki(Grafana system) or other Fluentds.
I was looking through github bugs and noticed some similar but different things and I am really really confused as to how to implement FluentD with a Securely enabled Elasticsearch flow.
The Error I keep getting is:
2022-04-08 15:39:03 +0000…
I use my custom fluentd plugin and it does not work with ubuntu20 but for other ubuntu version is no problem.
Here is my error
Traceback (most recent call last):
22: from /usr/sbin/td-agent:15:in `'
21: from /usr/sbin/td-agent:15:in…
Hello please can someone advise on following issue:
I have AKS kubernetes cluster and I am using EFK stack to get logs. Fluentd is running in pods.
I have nginx application pod for which I parse log into separate fields.
nginx log:
10.244.x.x - -…
I have installed a aws-es-proxy helm chart. I am using https://github.com/kokuwaio/helm-charts/tree/main/charts/fluentd-elasticsearch repo for created fluentd.
2020-10-07 04:34:38 +0000 [error]: unexpected error error_class=NoMethodError…
2020-09-16 04:47:49 | info | PHP | BACKEND | 01ARZ3NDEKTSV4RRFFQ69G5FAV | Login not successfully. Please try again
Above is my sample logging format which will be sent to fluentd, and I did regular expression as follow:
time_format…
I have a Nginx Ingress Controller in the K8S cluster that has the following log format (I took it from /etc/nginx/nginx.conf in the container):
log_format upstreaminfo '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent…
I am trying to decide an approach for logs processing in a EKS cluster. Idea is to use EFK. We thought we can use fluentd to push the logs to elastic search. But most of the blogs uses fluentd to send the logs to cloudwatch and then a lambda to send…
I'm using TCPSampler in Jmeter to do the performance test on fluentd's TCP input plugin.
But it always runs into 500 error:
Thread Name:Thread Group 1-1
Sample Start:2020-06-23 18:21:08 CST
Load time:2372
Connect Time:360
Latency:0
Size in…
I am studying docker fluentd. That is what I do not understand.
-v $(pwd)/fluent.conf:/fluentd/etc/fluent.conf \
-e FLUENT_CONF=fluent.conf \
I don't get the meaning of commands.
I am using Amazon EKS and I have a server (consider it as X ) which is connected to the control node using kubectl.
I am able to get the pod logs from the server X by running the following command.
kubectl logs -f podname -n=namespace
Now my goal…
This bash command "etcdctl get system config/log/timestamp" returns time type either UTC or System. Now i want to use this to convert time to same format. How can i do that?
I tried this but it failed td-agent running.
@type exec
…
I'm using FluentD (deployed as DaemonSet) to stream k8s app (containers) logs to elasticsearch.
i want extract specific keys from the 'log' key string in elastic such as logKey: ... or statusCode:.. (inside the 'log' key below)
Please see the log…
I am using the curl command to send an event to Splunk cloud, but I am getting timeout errors and host not found errors. I believe it is the host name that is the problem.
I have tried the "Server Name" from the About section on our Splunk cloud UI.…
In fluentd how do i parse this log and get fields like ip, method and severity by using grok pattern or json
{"log":"2019-08-09 06:54:36,774 INFO 10.2.1.200 [09/Aug/2019:06:54:36 +0000] \"GET / HTTP/1.1\" 200 205 \"-\"…
In my Mac OS, With fluentd I'm trying to read tcp events and write it to local directory.
Below is the td-agent.conf that I have created to open TCP port and writing to local.
@type tcp
@log_level…
