Stack Overflow
Stack Overflow

Questions tagged [fluent-bit]

Fluent-Bit (not to be confused with Fluentd), is an open-source, light weight data collector which can act as both an aggregator and forwarder. It allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Fluent bit links :

379 questions
2
votes
1 answer

FluentBit S3 upload with container name as key in s3

My log file name in s3 looks like kube.var.log.containers.development-api-connect-green-58db8964cb-wrzg5_default_api-connect-fa7cafd99a1bbb8bca002c8ab5e3b2aefc774566bb7e9eb054054112f43f1e87.log/ here I want to extract only container name from tag so…
Vikas Shaw
  • 155
  • 1
  • 8
2
votes
1 answer

fluentbit connection to fluentd refused

The issue I have been trying to use the fluent-operator to deploy fluentbit and fluentd in a multi-tenant scenario in EKS cluster. The goal is to collect logs with fluentbit and then forward to fluentd to process and send to OpenSearch. The logs are…
Kaio H. Cunha
  • 211
  • 2
  • 10
2
votes
1 answer

Fluentbit record_modifier how to remove key with slash?

I'm trying to remove not interested kubernetes metadata from logs what I'm parsing with fluentbit v. 2.0.5. After couple of nest filters with lift action, I'm trying to delete keys with slash in name of a key, still see it in elasticsearch Below…
pluk
  • 99
  • 4
2
votes
0 answers

FluentBit: Where does field in logs for pod come from?

I have k8s cluster with сonfigured fluent-bit logger. Configuration of fluent-bit: custom_parsers.conf: | [PARSER] Name docker_no_time Format json Time_Keep Off Time_Key time Time_Format…
Радик Фахриев
  • 21
  • 2
2
votes
1 answer

Fluentbit not sending EKS logs to S3

I've tried to send EKS logs to S3, but logs are not being pushed to S3. But, I can able to visualize logs in Elastic search. Below is the output section of my fb-configmap.conf file. output-elasticsearch.conf: | [OUTPUT] Name …
Naveen
  • 103
  • 1
  • 12
2
votes
3 answers

containerd multiline logs parsing with fluentbit

After shifting from Docker to containerd as docker engine used by our kubernetes, we are not able to show the multiline logs in a proper way by our visualization app (Grafana) as some details prepended to the container/pod logs by the containerd…
k.elgohary
  • 403
  • 8
  • 21
2
votes
0 answers

TCP logging on FluentBit from Python

Trying to export logs to a fluent-bit TCP input server, from a Python module. fluent-bit is setup to listen to a localhost port with a TCP server, and in Python the logging creates a SocketHandler logging handler, to redirect the logs on that…
Sarriman
  • 382
  • 5
  • 22
2
votes
1 answer

How to classify docker container logs by container name using Fluent-bit?

We are using Fluent-bit to process our docker container logs, I can use Tail to specify container log path, but there are multiple different containers in that log path /var/lib/docker/containers/*/*.log. The first '*' can be interpreted to many…
user2706071
  • 196
  • 9
2
votes
1 answer

Why I get the "failed to flush chunk" error in fluent-bit?

I am sending logs from my nestjs project to elastic search using fluent-bit. However, I get the following error periodically: [2022/06/14 21:43:18] [ warn] [engine] failed to flush chunk '1-1654871535.433259986.flb', retry in 858 seconds:…
Erika
  • 453
  • 8
  • 23
2
votes
0 answers

Fluentbit to GCS

I was wondering if Fluentbit had a Google Cloud Storage (GCS) output plugin? I found this fluent-bit-go-gcs repo on github but not sure if this is still something that's being maintained. Any examples of Fluentbit configs that output to GCS/other…
skarsky
  • 63
  • 1
  • 6
2
votes
1 answer

How to split log (key) field with fluentbit?

We are sending node.js code to OpenSearch using FluentBit. We are having issues because log key contains nested value as message. We need to split the values mentioned in the below log message - log- {"level":"info","message":"\"{\"method:\" GET…
Prateek Porwal
  • 21
  • 3
2
votes
1 answer

FluentBit setup

I'm trying to set up FluentBit for my EKS cluster in Terraform, via this module, and I have couple of questions: cluster_identity_oidc_issuer - what is this? Frankly, I was just told to set this up, so I have very little knowledge about FluentBit,…
TFaws
  • 193
  • 2
  • 4
  • 15
2
votes
1 answer

How to encrypt fluentd SQL plugin password?

Because of security reasons, we can't keep SQL authentication in plain text, is there a way to hide or encrypt passwords? I am getting bad documentation and bad support from the plugin site. Unfortunately I can't keep this data in environment…
Mysterious288
  • 365
  • 5
  • 24
2
votes
2 answers

Automatic log group creation in AWS cloudwatch using fluent bit in EKS

Is there a way to automatically ship the logs from EKS pods to cloudwatch using fluent bit whenever new pods come up ? I am able to send the logs by adding INPUT, FILTER, OUTPUT section in fluent-bit configmap whenever we need to deploy a new…
Satyaki
  • 31
  • 5
2
votes
1 answer

Why does Fluent-bit use the varlibdockercontainers volume?

There is an official manifest for the deployment of the daemonset https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-ds.yaml Line 49 defines the volume varlibdockercontainers. I don't…
Maksim
  • 197
  • 2
  • 12
1 2 3
…
25 26