Stack Overflow
Stack Overflow

Questions tagged [fluent-bit]

Fluent-Bit (not to be confused with Fluentd), is an open-source, light weight data collector which can act as both an aggregator and forwarder. It allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Fluent bit links :

379 questions
0
votes
0 answers

Append 2 fields for host value in Fluentbit running in k8s

Is it possible to append 2 fields as a value for host in fluentbit. The fluentbit runs as a Daemonset [FILTER] Name modify Match * Add host ${K8S_CLUSTER_NAME} The above has the host with value pertaining to the env var set for…
Sai
  • 1,790
  • 5
  • 29
  • 51
0
votes
1 answer

AWS EKS with Fargate send logs to logstash

Good afternoon. I need to collect application logs in an eks cluster using fargate, in a node group environment I use a fluentbit running with daemon set to collect this and send it to a logstash. But as fargate doesn't support the set daemon, I'm…
Felipe Pereira
  • 1
0
votes
1 answer

Why does the official helm chart for fluent-bit start 20 pods

I followed the official helm chart for fluent-bit and ended up with 20 pods, in a namespace. How do I configure it to use 1 pod? The replicaCount attribute in values.yaml is set to…
Simon Liu
  • 463
  • 5
  • 12
0
votes
1 answer

Getting 'cannot list resource "pods/var"' from kubernetes

I am trying to set up fluent-bit in a custom namespace. I took the following steps (lots ellided here for clarity): I created the serviceaccount like this: apiVersion: v1 kind: ServiceAccount metadata: name: k8s-logger namespace: efk Then I…
bsellersful
  • 35
  • 7
0
votes
2 answers

FluentBit not reading logs from shared PVC

I have a PersistentVolumeClaims(PVC) with ReadWriteMany (RWX) access mode. This PVC is clamed by 2 pods. In the second pod I have FluentBit reading the logs stored by the application pod in the shared PVC. The issue is FluentBit recognizes the…
Sunil
  • 429
  • 1
  • 9
  • 25
0
votes
1 answer

Fluent-bit, How can I use strftime in path

my log file name contains the current date, like my_log_210616.log and I need to tail the file in fluent-bit. I tried with, [INPUT] Name tail Path /var/log/my-service/my_log_%y%m%d.log [OUTPUT] Name stdout Match * but it doesn't watch the…
Expert wanna be
  • 10,218
  • 26
  • 105
  • 158
0
votes
1 answer

How can I get details of GKE fluentbit related error?

We just found some logs are missing from Stackdriver, We can use kubectl logs for listing the logs message but some of them are not send to Stackdriver logs for some reason. An example of a log entry that…
nvcnvn
  • 4,991
  • 8
  • 49
  • 77
0
votes
1 answer

Fluent Bit not saving any data on filesystem

I am new to fluent bit and currently doing a POC. I tried multiple things but couldn't make Fluent Bit save any data to filesystem. [SERVICE] flush 1 daemon Off log_level trace parsers_file parsers.conf plugins_file…
ujjwal
  • 428
  • 3
  • 16
0
votes
2 answers

FluentBit: add dynamic es index

I have a working fluent-bit:1.7 I need OUTPUT to Elasticsearch and create a dynamic index based on the k8s label = name. I want the following convention for the index: infra-${app_name}-yyyy.mm.dd Example: infra-mongodb-2021.01.01,…
mrk
  • 640
  • 8
  • 16
0
votes
1 answer

fluent-bit TCP json parser for large integers pegged at 32bit maxint

Given the following configuration file: [INPUT] Name tcp Listen 0.0.0.0 Port 5170 Format json [OUTPUT] Name stdout Match * running fluent-bit on Windows and sending the string {"key":…
jltrem
  • 12,124
  • 4
  • 40
  • 50
0
votes
1 answer

C# Serilog conditional logging if one fails?

We are in the analysis stage, right now we have an architecture where Serilog will write to a local file, and then there is a Fluentd that will pull the logs and dumps them to the elastic search. The reason for such architecture is to make sure that…
Mysterious288
  • 365
  • 5
  • 24
0
votes
1 answer

AWS Firehose is throttling records ingested by FluentBit, how to handle?

Environment: Kubernetes Cluster: EKS Logging Agent: FluentBit version 1.2 Destination for FluentBit: AWS Kinesis firehose delivery stream Fluentbit output plugin: amazon-kinesis-firehose-for-fluent-bit Description: We have a setup where a FluentBit…
Akshay Hiremath
  • 950
  • 2
  • 12
  • 34
0
votes
1 answer

eks fluent-bit to elasticsearch timeout

So I had a working configuration with fluent-bit on eks and elasticsearch on AWS that was pointing on the AWS elasticsearch service but for cost saving purpose, we deleted that elasticsearch and created an instance with a solo elasticsearch, enough…
night-gold
  • 2,202
  • 2
  • 20
  • 31
0
votes
1 answer

ECS: firelens container can not push logs to S3

I'm currently using ECS tasks (EC2 hosted) for a web application which use firelens Fluent Bit as log router. I set the output destination to S3 using this plug-in, but it gets an access error like this: [error] [output:s3:s3.3] PutObject API…
takbian
  • 11
  • 2
0
votes
1 answer

Fluent-bit log forwarder issue

I have application deployed on kubernetes container along with logger SideCar fluent/fluent-bit:1.0.6 using But logger container fails with below error. Since fluent.conf comes inside docker image does any know how to fix below error Error in…
user1591156
  • 1,945
  • 4
  • 18
  • 31
1 2 3
…
25 26