Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [fluentd]

Fluentd is an open source log aggregator.

Fluentd is a fully free and fully open-source log collector that instantly enables you to have a "Log Everything" architecture with 125+ types of systems.

Fluentd treats logs as JSON, a popular machine-readable format. It is written primarily in C with a thin-Ruby wrapper that gives users flexibility.

Fluentd's performance has been proven in the field: its largest user currently collects logs from 5000+ servers, 5 TB of daily data, handling 50,000 msgs/sec at peak time.

37 questions
0
votes
1 answer

Fluentd elasticsearch comprssion

I forward logs from app servers with fluentd (type tail) to elasticsearch server. Do not understand the following: Can I enable for reducing traffic the compression transmitted logs with type tail or another type?
artful
  • 31
  • 1
  • 7
0
votes
1 answer

How to split a single record to multiple records in fluentd?

I would like to ask regarding to fluentd. My fluentd versions is below. td-agent-2.1.5-0.x86_64 fluentd 0.10.61 I now have a tail input plugin using multiple line format which parses multiple lines from log and set as a single record as…
Yu Watanabe
  • 616
  • 2
  • 8
  • 18
0
votes
3 answers

Configuring Logrotate for Fluentd files. Necessary?

I have the following fluent.conf type forward type monitor_agent port 24220 # Listen DRb for debug type debug_agent port 24230 type tail path…
james_womack
  • 113
  • 1
  • 5
0
votes
0 answers

Fluentd logs not sent to Elasticsearch

I have a cluster in VirtualBox to learn kubernetes. I have a deployment that contains MySQL and phpMyAdmin. I created a DemonSet that has the fluentd image and collects the logs to transmit them to elastics at ip 10.0.2.11. I don't understand why it…
vasile Berco
  • 1
0
votes
1 answer

Could not communicate to Elasticsearch, resetting connection and trying again. EOFError (EOFError)[Fluentd using helm on kubernetes]

I'm installing elasticsearch, kibana and fluentd in kubernetes with helm chart Elasticsearch and kibana pods stand up smoothly, but fluentd pods don't stand up I get the following errors: 2023-06-13 13:29:39 +0000 [warn]: #0 [filter_kube_metadata]…
zoroglur
  • 101
  • 1
0
votes
0 answers

Discard less important logs during overload with fluentbit or fluentd

Assuming I could identify more or less important logs by pattern-matching them, is there a way to configure fluentd (or fluentbit) to do intelligent shedding (discards) when it starts to buffer to much (back-pressure from output)? Are there other…
Gregor
  • 541
  • 3
  • 13
0
votes
1 answer

How to enable read permission on Kubernetes mount path

I have installed FluentD on Kind-Kubernetes cluster on CentOS VM running on my laptop. I'm having issues getting FluentD to read logs as it is throwing the following error. 2021-08-29 08:26:31 +0000 [warn]: #0 [tail_container_logs]…
Jason Nanay
  • 1
  • 1
1 2
3