Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [fluentd]

Fluentd is an open source log aggregator.

Fluentd is a fully free and fully open-source log collector that instantly enables you to have a "Log Everything" architecture with 125+ types of systems.

Fluentd treats logs as JSON, a popular machine-readable format. It is written primarily in C with a thin-Ruby wrapper that gives users flexibility.

Fluentd's performance has been proven in the field: its largest user currently collects logs from 5000+ servers, 5 TB of daily data, handling 50,000 msgs/sec at peak time.

37 questions
1
vote
0 answers

Microsoft OMSAgent Listens on TCP port 16001

I have installed the Microsoft OMSAgent (on Linux). Now I see that it listens on TCP port 16001. Neither the documentation nor grepping through the Sourcecode gave some hints why. Anyone got an idea what services usually uses port 16001? fluentd?…
arved
  • 453
  • 2
  • 14
1
vote
1 answer

How to address container for logging using docker-compose

A container described in docker-compose.yml uses logging extension to send log to a fluentd container. version: "2" services: fluentd: image: fluent/fluentd:v0.14.8 container_name: fluentd nginx: image: nginx:1.11.5 …
Kartoch
  • 233
  • 4
  • 14
1
vote
0 answers

fluentd missing region error

i want to fetch cloudwatch logs. I installed cloudwatch input plugin and configured the config file of td-agent. but im getting the error as "unexpected error error_class=aws::errors::MissingRegionError". Any help on how to resolve this would be…
arunapavan
  • 11
  • 2
1
vote
1 answer

Fluentd not writing apache access log to mongodb

I'm following the instructions from this link to write apache access logs to mongo db- http://docs.fluentd.org/articles/apache-to-mongodb. But nothing is getting written to mongo DB. I'm checking using db["access"].findOne();. Contents of my…
ksrini
  • 121
  • 5
1
vote
1 answer

Installing fluentd / Kibana / Elastic Search on Suse Enterprise Linux 11

I have successfully installed the recommended fluentd stack ( fluentd, Kibana, Elastic Search ) on a Centos 6 virtual machine. But our production environment is all Suse Enterprise Linux 11. I am trying to work out how to get fluentd installed but I…
user35861
1
vote
2 answers

Rsyslog - "Warning: CA certificate is not set" but TLS forwarding still works

I am configuring rsyslog to forward over TCP/TLS. I was curious why, unlike HTTPS, I had to add the certificate to my client, so I went against the README and commented out the cert config, leaving me with this: #…
threewordphrase
  • 23
  • 4
0
votes
2 answers

Kubernetes DaemonSet Pod exceeds memory limit but is not killed

We use kubernetes (specifically: openshift) to run our infrastructure in our team. One of the daemonsets, fluentd, is currently causing a lot of trouble, frequently taking full nodes down with huge CPU, memory and disk I/O requirements (like really,…
aspyct
  • 360
  • 1
  • 6
  • 21
0
votes
1 answer

installing google-fluentd agent on docker produce an error

I am trying to install google-fluentd agent on a docker and get the following error: Restarting google-fluentd: google-fluentd ... failed! The OS is Debian GNU/Linux 9 (stretch) Docker file snippet: RUN cd /tmp && curl -sSO…
Erez Ben Harush
  • 177
  • 1
  • 7
0
votes
1 answer

Parsing or Reformatting Logs before feeding them to Splunk or Elastic Search

I have very complex log messages, that I want to reduce to the most important fields in order to save quota. The log messages are multiline and there is a lot of redundant information in them. A solution is to script something to reformat these logs…
gspoosi
  • 131
  • 4
0
votes
1 answer

Send Kube logs to Cloudwatch using fluentd

My cluster is on AWS and I've used kops to build by cluster. I'm trying to send the kubernetes logs to AWS Cloud-watch using fluentd. I was able to stand-up the fluentd pods. I am seeing the below error from the pod's logs. I have created…
sumanth
  • 27
  • 4
0
votes
2 answers

systemd and launching docker containers

So I am new to docker and wanted to start up a logspout docker container on server start up so I thought i would just chuck in a systemd start up file: [Unit] Description=Logspout GELF…
SnazzyBootMan
  • 101
  • 2
0
votes
2 answers

td-agent shared key mismatch between servers

I am trying to set encrypted logs with td-agent between two machines. I have followed the examples from the official documentation of fluentd however i am stuck in a very weird situation. The client and server refuse to negotiate due to shared_key…
giomanda
  • 1,754
  • 4
  • 21
  • 30
0
votes
2 answers

How to config Zabbix item read data from port and create graphs?

I am new to Zabbix. I send data from fluentd client to zabbix server port 10050. My question is How should I config Zabbix item to read this data from port and generate graph for it. My data is…
sam
  • 101
  • 5
0
votes
0 answers

fluentd services die immediately after they start up

Please excuse the naivety in my question, but this is not a subject I know much about at present. My company is currently running kubernetes-managed fluentd processes to push logs to logstash. These fluentd processes start up and fail immediately…
Paul Pritchard
  • 111
  • 5
0
votes
0 answers

using google fluentd to log from compute engine instance in google cloud

I got a problem with the logging agent from Google Cloud. What i wanted was to use the google-fluentd (based on fluentd) on a webserver to collect the logs and sent it to Stackdriver Logging Api. The server itself is a Google Compute Instance, so…
Computerui
  • 3
  • 2
1
2
3