Stack Overflow
Stack Overflow

Questions tagged [fluent-bit]

Fluent-Bit (not to be confused with Fluentd), is an open-source, light weight data collector which can act as both an aggregator and forwarder. It allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Fluent bit links :

379 questions
1
vote
0 answers

add custom metrics to fluent bit exec plugin

I have a script that running via exec input plugin of fluent bit. I want to add custom metrics from the script execution to the original /api/v1/metrics/prometheus API. What I need to add to my script in order to publish custom metrics ?
Shurik
  • 562
  • 1
  • 7
  • 19
1
vote
2 answers

"[error] [upstream] connection timed out after 10 seconds" failed when fluent-bit tries to communicate with fluentd in Kubernetes

I'm using fluent-bit to collect logs and pass it to fluentd for processing in a Kubernetes environment. Fluent-bit instances are controlled by DaemonSet and read logs from docker containers. [INPUT] Name tail Path…
AnujAroshA
  • 4,623
  • 8
  • 56
  • 99
1
vote
0 answers

Issues with lifting stringified JSON

need some help with fluentbit parsing.. really struggling here. I've got an entry that currently looks like that in the stdout plugin: [0] datapoints: [1635420146.768714629, {"log"=>"{"@timestamp": "2021-09-29T06:36:48.989Z", "@version": 1,…
Shai Katz
  • 1,603
  • 12
  • 22
1
vote
2 answers

fluent bit kubernetes annotation does not work

I deploy fluent bit to kubernetes. And I deploy one pod with annotation fluentbit.io/parser: cri. But it still parse the log with the parser ivyxjc which is config in INPUT. fluent-bit config apiVersion: v1 kind: ConfigMap metadata: name:…
fei_hsueh
  • 161
  • 1
  • 11
1
vote
0 answers

How to pipe processing fields by regex and json in Google Ops Agent?

/etc/google-cloud-ops-agent/config.yaml logging: receivers: app: type: files include_paths: [/www/app-*.log] processors: monolog: type: parse_regex field: message regex:…
Gander
  • 1,854
  • 1
  • 23
  • 30
1
vote
2 answers

kubernetes fluent-bit unable to resolve host

Issue Description I am getting an error in fluent-bit basically saying it cant resolve host getaddrinfo(host='.aws.elastic-cloud.com:9243', err=4): Domain name not found I suspect it has something to do with the port…
Tim Holum
  • 697
  • 1
  • 11
  • 24
1
vote
2 answers

Fluency with forward plugin: how to add kubernetes metadata to logs

Hey i have a question. Im using logback-more-appenders(fluency plugin) to send logs to EFK stack (fluent-bit) which is working in kubernetes cluster, but it lacks kubernetes metadata ( like node/pod names). I know i can use…
Randers
  • 28
  • 6
1
vote
1 answer

How to handle STDOUT logs in K8s?

In a Docker environment my Java-App logs on STDOUT via log4j, the messages will be sent to a Graylog instance. There is no special logging config besides configuring the Console-Appender to use JsonLayout. My docker-compose.yml snippet: logging: …
O. Schnieders
  • 593
  • 1
  • 6
  • 16
1
vote
0 answers

Splunk Kubernetes example

I found this entry Send Kubernetes pod's logs to Splunk I was wondering if anyone had a front-to-back example (blog, video, etc.). I'm used to Kubernetes, but not used to Splunk. I want to deploy something like the fluentbit with fluentd solution or…
el n00b
  • 1,957
  • 7
  • 37
  • 64
1
vote
2 answers

Fluent Bit cannot reach DNS for Azure Log Analytics

I'm running Fluent Bit on Azure Kubernetes Service as DaemonSet, output goes to Azure Log Analytics. Once Fluent Bit tries to send logs it fails to connect DNS server: getaddrinfo(host='XXX.ods.opinsights.azure.com', err=12): Timeout while…
Michael Perlin
  • 11
  • 4
1
vote
1 answer

Fluentbit rewrite_tag not working with JSON Array

We are using fluent-bit plugin to tail from a file and send to an HTTP endpoint. The sample log looks like the following. tenant 1 testing 100 The configuration for input looks like the following. [INPUT] Name tail Path…
Subhankar
  • 487
  • 8
  • 25
1
vote
0 answers

EFK-Stack, can't configure index lifecyle management properly

i am running an EFK-Stack (elastic, fluent-bit, kibana) on an azure kubernetes service. For some reasen i dont get the index lifecyle management to work properly. I added the Logstash_Format On to the output-elasticsearch.conf in order the create a…
Martin Hering
  • 11
  • 3
1
vote
1 answer

GKE Fluent bit partial logs

I have K8S cluster in GCP (version is 1.20.8-gke.900 from the regular update channel). All cluster pods write logs in STDOUT or STDERR from Docker containers. A couple of weeks ago we found that some log entries are missing in the GCP logging…
Viktor Kurchenko
  • 35
  • 8
1
vote
2 answers

Kubernetes logs not found in default locations?

In my k8s environment where spring-boot applications runs, I checked log location in /var/log and /var/lib but both empty. Then I found log location in /tmp/spring.log . It seems this the default log location. My problem are How kubectl log knows…
Viraj
  • 5,083
  • 6
  • 35
  • 76
1
vote
2 answers

Fluentbit parse json

I'm using fluentbit to parse logs. I have logs in next format: {"key1":"value1","key2":"{\n \"date\": \"2021-07-05 13:58:20.501636\",\n \"timezone_type\": 3,\n \"timezone\": \"UTC\"\n}", "key3":"{ \n \"somedata\": \"somevalue\" "} In ES and…
ivanovUA
  • 79
  • 6
1 2 3
…
25 26