Stack Overflow
Stack Overflow

Questions tagged [fluent-bit]

Fluent-Bit (not to be confused with Fluentd), is an open-source, light weight data collector which can act as both an aggregator and forwarder. It allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Fluent bit links :

379 questions
1
vote
1 answer

How to remove "timestamp" date from the Fluent-Bit logs?

I'm testing Fluent-bit for my local k8s cluster which has a CRI runtime interface and I'm sending logs to a slack channel. But the problem is that Fluent-Bit is assigning a "timestamp" in the log and I'm not able to remove it. Maybe someone knows a…
Gevegeve
  • 21
  • 3
1
vote
0 answers

Access the keys of an attribute with Map type in Opentelemetry Collector

I am implementing Opentelemetry Collector in Kubernetes. I have a fluentbit set up for log forwarding. Logs are forwarded to OpenTelemetry fluentforward endpoint and I need to parse the log data and export it to Loki. **The goal: ** add labels to…
akd
  • 11
  • 1
1
vote
0 answers

Fluentd to OpenSearch: How to create dynamic indexes with K8s metadata?

I'm using the fluent-operator to deploy fluentbit and fluentd. Fluentbit collects and enriches the logs with Kubernetes metadata, then forwards to Fluentd. Fluentd ships the logs to AWS OpenSearch. I was trying to add a namespace label to the…
Kaio H. Cunha
  • 211
  • 2
  • 10
1
vote
0 answers

Dynamically create output folder for each module logs using deployment name in fluentBit

I've several modules deployed in my kubernetes cluster and I use fluentBit to read logs for all modules from /var/log/containers folder and write them to a file on a persistent volume. I'm able to successfully write all log files to a single folder…
Omar Raghib
  • 11
  • 1
1
vote
1 answer

Rancher fluientd: Error response from daemon: Duplicate mount point: /var/lib/docker/containers

I have a problem with the rancher-logging-root-fluentbit daemon. Some pods start correctly, others have this error. Error: Error response from daemon: Duplicate mount point: /var/lib/docker/containers Any ideas to solve this error? Thanks. I have…
Marco Brunet
  • 54
  • 6
1
vote
1 answer

How to assign the log path to "source" field in Splunk while using fluent-bit?

I am using fluent-bit as the log collector and redirecting the log files to Splunk HEC. In event metadata, I would like to set "source" as the log path but it failed. [INPUT] Name tail Tag jsm_on_centos8-2 Path …
joker57
  • 110
  • 7
1
vote
1 answer

How to use an inner record field as message key in Fluent Bit Kafka output?

With the config below: [INPUT] Name tail Path /var/log/containers/*.log multiline.parser docker, cri Tag kube.* Mem_Buf_Limit 5MB Skip_Long_Lines On [FILTER] Name kubernetes Match kube.* Merge_Log On Keep_Log…
kafka-user122
  • 11
  • 3
1
vote
0 answers

Fluent-bit parsing json in side of a string

I have a k8s logging stack set up like this: fluent-bit => fluentd => elastic I have a working set up with this config: fluent-bit.conf [SERVICE] Daemon Off Parsers_File custom_parsers.conf HTTP_Server On HTTP_Listen 0.0.0.0 Health_Check…
SolidQ
  • 19
  • 2
1
vote
0 answers

Issue with custom log routing with ECS fargate, Firelense and fluentbit to cloudwatch

I am trying to get logs from my app container to cloudwatch using firelesne and fluentbit by aws, and not getting it. Application writes log on /opt/app/log/*.log here is my task definition and fluentbit config file. ` resource…
Nishant
  • 11
  • 4
1
vote
0 answers

Fluent Bit docker input plugin doesn't output metrics in kubernetes (AWS EKS)

I have a Kubernetes cluster running in AWS (EKS) with some pods running in it. I would like to use Fluent Bit to fetch (not only logs) but also metrics from the pods/containers. The logging works fine (I use the tail input plugin for that). But the…
hfm
  • 567
  • 2
  • 7
  • 24
1
vote
1 answer

FluentBit into Azure Log Analytics Workspace memory buffer

I have a client application that runs in AKS with a pod that forwards the logs to Log Analytics Workspace. The software is from a third party that they purchased and we are spitting out Log4J2 logs using the JSON formatter (a custom one, not the…
el n00b
  • 1,957
  • 7
  • 37
  • 64
1
vote
2 answers

OpenSearch 401 for /_bulk

I am using fluent bit to stream logs from Kubernetes to OpenSearch (AWS). I have deployed via the Helm charts and have configured the output as below [OUTPUT] Name opensearch Match * Host aws-domain-name.region.es.amazonaws.com Port…
Shaun
  • 475
  • 1
  • 5
  • 16
1
vote
0 answers

fluent-bit: [error] [input:tail:tail.0] read error, check permissions: /var/log/containers/*.log" in Azure Red Hat Openshift

I'm newguy in Openshift, and trying to install fluent-bit to collect logs from applications. I've installed fluent-bit via this helm using this documentation. daemonSet starts and looks good but in logs of fluent-bit I see [2022/08/24 15:33:56] […
Konstantin GidMaster Syrovatsk
  • 21
  • 4
1
vote
0 answers

How do I set the number of logs to be written at a time

I use fluent-bit input from systemd and out put to clickhouse.And I code the plugin by golang. example: func FLBPluginFlushCtx(ctx, data unsafe.Pointer, length C.int, tag *C.char) int { dec := output.NewDecoder(data, int(length)) batch, err…
zhang_
  • 21
  • 1
  • 4
1
vote
0 answers

Kubernetes pod name/service name as index name in Kibana using fluentd

Currently we have many services running on k8s and sending logs with fluent-bit to elastic using fluentd. In fluentd we have hard coded logstash_prefix xxx-logstash, so all logs are created with the same index. Now we want to send data to elastic…
Raushan-cs
  • 39
  • 6
1 2 3
…
25 26