Stack Overflow
Stack Overflow

Questions tagged [fluent-bit]

Fluent-Bit (not to be confused with Fluentd), is an open-source, light weight data collector which can act as both an aggregator and forwarder. It allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Fluent bit links :

379 questions
4
votes
1 answer

Fluentbit nested json parsing

I have the following log { "log": { "header": { "key": "value", "nested": "{\"key1\":\"value\",\"key2\":\"value\"}", "dateTime": "2019-05-08T20:58:06+00:00" }, "body": { "path": "/request/path/", …
Reda Drissi
  • 1,532
  • 3
  • 20
  • 32
4
votes
1 answer

Fluent Bit 1.8+ and MULTILINE_PARSER

My goal is to collect logs from Java (Spring Boot) applications running on Bare Kubernetes. These logs are then translated into ES and visualized in Kibana. For these purposes I deployed Fleunt Bit 1.8.9 via Kubernetes 1.22. Since I use Containerd…
Maksim
  • 197
  • 2
  • 12
4
votes
0 answers

EKS Fargate Fluent-Bit multiple Outputs

I'm running a K8 cluster on Fargate and using FluentBit to send logs to cloudwatch https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html#fargate-logging-troubleshooting I'm able to send all logs to a cloudwatch group with this…
Tony
  • 656
  • 8
  • 20
4
votes
2 answers

Fluent Bit Tail Input only reads the first few lines per log file until it is restarted again

The Problem I have a Fluent Bit service (running in a docker container) that needs to tail log files (mounted from the host into the container) and then forward those logs to Elasticsearch. For this PoC I create a new log file every minute (eg.…
Dewald
  • 41
  • 1
  • 2
4
votes
0 answers

Does Fluent Bit Input plugin "forward" support multi-line logs processing?

I trying to solve an issue with merging Java stack trace messages into one log event with a Fluent Bit forward plugin. Original logs are written as plain text with separators (not JSON format), so I have to deal with multiline issue. Logs are…
Leonid Samarkin
  • 186
  • 1
  • 7
4
votes
0 answers

Fluentbit http output - send one log per request

I'm trying to get fluenbit (td-agent-bit-1.6.3-win64) to send log lines from a file to a HTTP endpoint. My problem is, that the endpoint can only accept 1 JSON object by POST at a time, and it looks like fluentbit always sends several JSON objects…
yorondevops
  • 61
  • 3
4
votes
2 answers

fluentbit reporting each line of the java stacktrace log as new log event

any help is greatly appreciated. I have fluentd running on kubernetes. I have logs coming from java application. The stack trace is multiline and the issue is each line of the multiline log is reported as a new log event. below is my config. can…
Ishvar
  • 51
  • 1
  • 3
4
votes
1 answer

Adding a label to filer log entries in Loki

I have created simple script to test Grafana Loki. It sends messages over Fluentbit: from fluent import sender messages = [ {'from': 'userA', 'to': 'userB', 'log': 'Hello!'}, {'from': 'userB', 'to': 'userA', 'log': 'Hi!'}, ] for message in…
Roman Newaza
  • 11,405
  • 11
  • 58
  • 89
4
votes
1 answer

Is there any possible to install bash in distroless image?

I'm working on Fluent-bit I need to execute entry-point script in the container but fluent-bit image is distroless base image so any possibilities are there to install bash or any idea for executing that script? please let me know
HARINI NATHAN
  • 217
  • 3
  • 12
4
votes
0 answers

How to upload all logs in directory recursively with Fluent Bit?

Using Fluent Bit to upload directory: [INPUT] Name tail Path /var/log/* Only files directly under /var/log/ are handled, but files in sub-directory are not handled. I've also tried using the ** syntax, but Fluent Bit doesn't…
speller
  • 1,641
  • 2
  • 20
  • 27
4
votes
1 answer

Out of order logs between FluentD and FluentBit

I'm currently using JournalD + JournalBeat + Logstash as logging stack but I want to switch to using JournalD + FluentD. I tried using https://github.com/fluent-plugin-systemd/fluent-plugin-systemd as a input for FluentD but it results in a low…
Jonathan Erlich
  • 41
  • 3
4
votes
2 answers

Fluentbit with mycat multiline parsing

I want make a log management system through EFK. I'm trying for days now to get my multiline mycat log parser to work with fluent-bit. My fluentbit configuration: parsers.conf: [PARSER] Name mycat_error_log_parser_head Format…
dream
  • 51
  • 1
  • 1
  • 4
3
votes
0 answers

Fluent Bit only receiving logs on first connection

I'm trying to use Serilog (the Serilog.Sinks.Network sink) to write to a Fluentbit instance (just running locally). I'm finding that if I start Fluentbit, then run my app, the logs get processed as expected. However, without restarting Fluentbit, if…
Dan
  • 5,692
  • 3
  • 35
  • 66
3
votes
1 answer

GKE: Classify log messages based on tag in message payload

I have a third-party go based app that is writing logs to stderr. Even though the log message is written as an INFO message stackdriver classifies it as ERROR because the log message is written to stderr (which seems to be default for golang…
RandomQuests
  • 635
  • 4
  • 16
3
votes
2 answers

Fluent Bit does not send logs from my EKS custom applications

I am using AWS Opensearch to retrieve the logs from all my Kubernetes applications. I have the following pods: Kube-proxy, Fluent-bit, aws-node, aws-load-balancer-controller, and all my apps (around 10). While fluent-bit successfully send all the…
nolwww
  • 1,355
  • 1
  • 15
  • 33
1
2
3
…
25 26