Fluentd is open-source and distributed data collector, which receives logs in JSON format, buffers them, and sends them to other systems like Amazon S3, MongoDB, Hadoop, Loki(Grafana system) or other Fluentds.
I'm trying to send mongo metrics to statsd using fluentd but I'm going through some problems.
This is my conf:
type serverstatus #https://gist.github.com/chrischang12/943a69b02f3435281557
uri mongodb://user:pass@localhost:27017/admin
…
how can I write the regex for the following nginx access log format in fluentd ?
log_format main '$remote_addr - $remote_user [$time_local] $request '
'"$status" $body_bytes_sent "$http_referer" '
…
We load logs from apache access log file with fluent in_tail plugin and load into mongodb with the out_mongo plugin. We have about 10 different kinds of log messages in the log file. Would ideally like to keep them in separate mongo collections so…
I'm trying to collect syslog messages from cisco equipment and send it to sensu for analysis and alerting. Is it possible/feasible with fluentd? If yes, could you please advise the way?
Thank you,
Alexei
Sorry in advance if it sounds very naive.
I started working with Elastic Search, I have about 10GB data coming in that i wanted to store in Elastic Search everyday, I read Elastic search does compression to 5% of original data.
I have set up…
For example, consider the fluentd project: https://github.com/fluent/fluentd/blob/master/bin/fluentd
After I have perform a git clone, I can run the server inside the bin folder, e.g.
./bin/fluentd
And in this file, I can see it will load the lib…
I'm currently doing some research on async event logging projects. In the documentation of fluentd, I've seen the built-in in_forward plugin. I'm trying to use it with php event logger of fluentd. I'm trying to forward the local logs into a central…
I am facing an issue with FLUENTD gem installation.
Issue::
C:\RailsInstaller\Ruby1.9.3\bin>gem install fluentd-0.10.32.gem
Temporarily enhancing PATH to include DevKit...
Building native extensions. This could take a while...
ERROR: Error…
I am parsing my database logs using fluentd where I am parsing the logs using regular expression and indexing them to solr, I have a problem where few logs are in the below format
2023-08-24 08:28:20.278 UTC [26154] abcd@rst STATEMENT:SELECT
…
I'm running elasticsearch, fluentd, and Kibana in an EKS Kubernetes cluster. I have 3 pods in the same cluster that generate logs which are written to a volume mount, which is mounted in the fluentd pod. I have provided all 3 log paths in the fluent…
I am trying to install fluentd and having configuration below:-
Docker Version=20.10.12
Alpine Linux Version=3.17.3
Ruby Version= 3.1.4p223
I am installing the fluentd with the script details below:-
apk add sudo
sudo apk update
sudo apk add…
I am trying to create a dockerfile that will have image 1 and image 2 pulled from dockerhub. It seems that second image is overriding the changes of first image. How to handle this scenario.
FROM clamav/clamav:1.1
COPY clamd.conf /etc/clamav
EXPOSE…
Holla amigos, In Google Cloud GKE I have 3 containers inside a pod. The first one is the application, the second is the istio-proxy sidecar, and the third one is the fluentd sidecar. The scenario is simple where I would like to block/stop the logs…
Trying to send Kafka server logs from Rsys to FLuentd.
First converted the Kafka server log into Json, and then sending the Json logs from rsys.
VM1: Kafka,Rsyslog installed
VM2: FuentD
Example of Kafka server logs.
{"timestamp":"2022-07-29…
I'm unable to push the below logs via rsyslog. The rsyslog is only forwarding one line of the log.
Kafka-server logs:
[2022-07-25 11:43:45,091] INFO KafkaConfig values:
advertised.host.name = null
advertised.listeners =…
