1

My Fluent Bit Docker container is adding a timestamp with the local time to the logs that received via STDIN; otherwise all the logs received via rsyslog or journald seem to have a UTC time format.

I have a basic EFK stack where I am running Fluent Bit containers as remote collectors which are forwarding all the logs to a FluentD central collector, which is pushing everything into Elasticsearch.

I've added a filter to the Fluent Bit config file where I have experimented with many ways to modify the timestamp, to no avail. It seems like I am overthinking it; it should be much easier to modify the timestamp.

These are all the ways I've tried to modify the timestamp with the fluent-bit.conf filter

[FILTER]
    Name         record_modifier
    Match_Regex  ^(?!log.*).*$         ## only match the input received via stdin
    Tag          log.stdout            ## tag to mark input received via stdin 
    Add          sourcetype timestamp  ## tried to add timestamp from lua script 
    Parser       docker                ## tried to use docker parser for timestamp
    Time_key     utc                   ## tried to add timestamp as a key
    script       test.lua              ## sample lua script from fluentbit docs
    call         cb_print              ## call a function from within lua script

What is the de facto method to make all the timestamps uniform to UTC? Any help or suggestion is appreciated.

Shōgun8
  • 482
  • 10
  • 20

1 Answers1

4

The way it works is that the docker parser extracts the content of 'log' and respect the timestamp defined by docker.

One quick workaround would be to modify your parsers.conf and make sure the docker parser does not resolve the timestamp, on that way Fluent Bit will assign the current time in UTC for you.

edsiper
  • 398
  • 1
  • 4