How to parse a specific message and send it to a different output with fluent bit

Question

I need to parse a specific message from a log file with fluent-bit and send it to a file. All messages should be send to stdout and every message containing a specific string should be sent to a file. I have managed to do it with a filter with the following configuration

[SERVICE]
    Flush     1
    Log_Level info

[INPUT]
    Name        tail
    Path        inputfile.log
    Tag         test_tag

[FILTER]
    Name          rewrite_tag
    Match         test_tag
    Rule          $log (user_temporarily_disabled)  from.$TAG.new true
    Emitter_Name  re_emitted

[OUTPUT]
    Name   stdout
    Match  test_tag

[OUTPUT]
    Name file
    Match from.*
    File myoutput.log

With the following configuration whenever i send a line to the input file it goes to stdout in any case and it goes to file if the line contains the "user_temporarily_disabled" string. This is achieved by rewriting the tag with the rewrite_tag filter.

What i need more is to parse the message and rewrite it a new form. I have tried to add a Parser with no success

score 2 · Answer 1 · answered Aug 28 '20 at 09:13

ok i found it after spending some time

[SERVICE]
    Parsers_File parserFile.conf

[INPUT]
    Name        tail
    Path        inputfile.log
    Tag         inputtag

#first filter to redirect to parser
[FILTER]
    Name parser
    Match inputtag*
    Key_Name log
    Parser myparser

#second filter to rewrite tag after parser
[FILTER]
    Name          rewrite_tag
    Match         *
    Rule          $ALARMTEXT (user_temporarily_disabled)  newtag true
    Emitter_Name  re_emitted

[OUTPUT]
    Name file
    Match newtag*
    File output.log

[OUTPUT]
    Name stdout
    Match *

and the parser should be something like this

[PARSER]
    Name myparser
    Format regex
    Regex ^(?<INT>[^ ]+) (?<FLOAT>[^ ]+) (?<BOOL>[^ ]+) (?<ALARMTEXT>.+)$

now if i send something like this to the input file :

echo "111 0.1 true user_temporarily_disabled" >> inputfile.log

it goes to the file AND the output. Anything not parsed goes to the output only

How to parse a specific message and send it to a different output with fluent bit

1 Answers1