Stack Overflow
Stack Overflow

Questions tagged [fluent-bit]

Fluent-Bit (not to be confused with Fluentd), is an open-source, light weight data collector which can act as both an aggregator and forwarder. It allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Fluent bit links :

379 questions
1
vote
1 answer

Time between successive retries in Fluent Bit

There is a property called Retry_Limit but how to set time between successive retries? [OUTPUT] Name es Host 192.168.5.20 Port 9200 Logstash_Format On Retry_Limit 5
ujjwal
  • 428
  • 3
  • 16
1
vote
1 answer

How to parse a fluent-bit json $log. key?

I have the following log to be parsed: TID: [-1234] [] [2021-05-31 09:53:26,680] - Unique ID: Evento_Teste, Event: {"event":{"metaData":"blue"}} And below the configuration files that I created to deal with it. #td-agent-bit.conf [SERVICE] …
Kleyson Rios
  • 2,597
  • 5
  • 40
  • 65
1
vote
1 answer

FluentBit in GKE Autopilot or alternatives?

I'm migrating my cluster to GKE using autpilot mode, and I'm trying to apply fluentbit for logging (to be sent to Elasticsearch and then Kibana to be alerted on a slack channel). But it seems that GKE Autopilot doesn't want me to do anything on the…
MatsuzakaSteven
  • 197
  • 1
  • 16
1
vote
1 answer

EKS pods logging to Elastic Cloud

I am trying to set up pods logs shipping from EKS to ElasticSearch Cloud. According to Fluent Bit for Amazon EKS on AWS Fargate is here, ElasticSearch should be supported: You can choose between CloudWatch, Elasticsearch, Kinesis Firehose and…
Alexey Raga
  • 7,457
  • 1
  • 31
  • 40
1
vote
0 answers

Logging for AWS ECS FARGATE

I'm trying to use AWS Firelens for our FARGATE container logs. I'm are doing so by using the side-car method. Everything is working as expected. However I have one concern, i.e. if the logs in the source container is rotational. In other words,…
Red Bottle
  • 2,839
  • 4
  • 22
  • 59
1
vote
0 answers

fluentbit logs are creating over 1000+ streams, how to merge logs if same application but new podname

Currently in our environment on EKS we have fluentbit to collect logs and push to Cloudwatch, we noticed that same service that has been redeployed after new code changes that a new log stream is created, this causes over 1000 log streams, we are…
Alwandy
  • 11
  • 1
1
vote
1 answer

Handling exception stack trace while writing serial no in json logs using Json template layout log4j2

I am using json template layout for writing json logs for my ecs service to s3 via kinesis firehose and firelens. Following is the configuration for json template layout I am using - { "context_map": { "$resolver": "mdc" }, …
AMAN GUPTA
  • 121
  • 2
  • 10
1
vote
1 answer

Fluent Bit set index mapping

I am trying to set all mapped fields to string ie if a json message comes with following: { "logDate": "2012-04-23T18:25:43.511Z", "logId": 123131, "message": { "username": "pera", "password": "pera123" } } I need to…
Milos Zivkovic
  • 107
  • 6
1
vote
1 answer

Fluentbit does not parse square brackets

I am trying to parse following line 2021-03-09 05:31:41.396 [main] INFO [][][] o.s.c.b.c.PropertySourceBootstrapConfiguration - Located property source: [BootstrapPropertySource {name='bootstrapProperties-configmap.aa.default'}] with this…
Łukasz
  • 1,980
  • 6
  • 32
  • 52
1
vote
1 answer

Fluent-bit Variables in Key configuration

I'm creating a custom Fluent-Bit image and I want a "generic" configuration file that can work on multiple cases, i.e.: it should work with a forward input sometimes and with tail input some other times. I thought about using environment variables…
Jérôme B
  • 311
  • 5
  • 18
1
vote
1 answer

How to send EKS (fargate)container logs to cloud watch using fluent bit?

I have going through documentation https://aws.amazon.com/blogs/containers/fluent-bit-for-amazon-eks-on-aws-fargate-is-here/ and implementing it. But looks like it is not working. I have created IAM policy & attached to Pod execution (cluster…
Vishnu
  • 97
  • 1
  • 13
1
vote
0 answers

error io_tls flb_io_tls.c:359 NET - Connection was reset by peer

[2020/12/10 11:44:42] [ warn] net_tcp_fd_connect: getaddrinfo(host='https://abc.amazonaws.com'): Name or service not known [2020/12/10 11:44:42] [error] [io_tls] flb_io_tls.c:359 NET - Connection was reset by peer [2020/12/10 11:44:42] [ warn]…
Paresh Sahoo
  • 127
  • 1
  • 2
  • 6
1
vote
1 answer

How to set InfluxDB custom measurement name When I use Fluent-bit Tail plugin with Kubernetes filter

When I use InfluxDB output plugins with Fluent-bit, I would like to set custom InfluxDB measurement name like the kubernetes. but, current measurement name that kubernetes logs stored is each log file name…
SoHyang Jeon
  • 11
  • 1
1
vote
1 answer

Only collect logs with specific field by fluent-bit

For example, my logs look like: { "fieldA":"logA", "foo":"bar" } or { "fieldA":"logA", "fieldB":"logB" } In my case, I only want logs with field "foo" to be collected, while dropping other logs.
Haoyuan Ge
  • 3,379
  • 3
  • 24
  • 40
1
vote
1 answer

How to overwrite a rewrite_tag in fluent-bit

In my scenario, I'm using a fluent-bit (version 1.6.1) filter to identify a particular log pattern and tag (from.solr.out) it. [FILTER] Name rewrite_tag Match test_tag Rule $msg "^\d{4}-\d{2}-\d{2}…
AnujAroshA
  • 4,623
  • 8
  • 56
  • 99
1 2 3
…
25 26