0

I need to create a new field 'status' if the log field contains a specific string. I tried below code in fluentd but this doesnt work. I need to check if the log field contains the string 'error:' then the new field status should have error else if it has ok it should have ok.

<filter **>
  @type record_transformer
  enable_ruby true
  <record>
    status "${\
      if record['log'].downcase.include? 'error:'
        puts 'error'
      elsif record['log'].downcase.include? 'ok:'
        puts 'ok'
      end}"
  </record>
</filter>

Can we use regexp to do this?

I also tried using scan.

<filter **>
  @type record_transformer
  enable_ruby true  
  <record>
   status ${record["log"].scan(/^.* ([[:<:]]error[[:>:]]|[[:<:]]ok[[:>:]]):.*$/i).first.compact} 
  </record>  
</filter>
Ankita Chowdhury
  • 33
  • 1
  • 6

2 Answers2

0

I don't have much experience with fluentd other than reading their documentation now. But can you please check the following code snippet?

<filter **>
  @type record_transformer
  enable_ruby true
  <record>
    status "${record['log'].downcase.include?('error:') ? 'error' : (record['log'].downcase.include?('ok:') ? 'ok' : '')}"
  </record>
</filter>

I doubt puts on your snippet will work. So I kind of changed a little bit in the code. Let me know if this is something you are looking for.

Naveen Honest Raj
  • 120
  • 8
  • THanks I will try this. But I read somwhere that include? doesnt work on strings having spaces. Is that true? I also tried an alternative using regexp. which I pasted above. But that isnt working either. – Ankita Chowdhury Jul 09 '21 at 20:30
  • I tried the above code but that doesnt seem to work. What worked is @type record_transformer enable_ruby true status ${record["log"].scan(/^.* ([[:<:]]error[[:>:]]|[[:<:]]ok[[:>:]]):.*$/i).first.compact} Only problem is its giving fluentd error of undefined method for compact for nil – Ankita Chowdhury Jul 09 '21 at 21:43
  • Can you please share an example of some log that could possibly present in record['log'] - so that I can have some more context. – Naveen Honest Raj Jul 09 '21 at 23:56
  • It will be generic log like ```2021-03-31 12:12:05.856 LOG : Message eRror: 2021-03-31 12:12:05.856 LOG : Message Error: 2021-03-31 12:12:05.856 LOG : Message > 2021-03-31 12:12:05.856 LOG : Message ok: ``` – Ankita Chowdhury Jul 10 '21 at 01:49
  • Hey, sorry! I am not sure how well I can help with this. Regex is my nightmare. May your regex scan returns null because some log isn't matching at all? Maybe try `.first&.compact` and try if things are showing up as expected? – Naveen Honest Raj Jul 10 '21 at 03:43
0

You are trying to make status ok or error, using fluentd plugin record_transformer. You no need to use puts to assign the value to the variable. Below is the snippet that might help.

<filter abc.logs>
@type record_transformer
enable_ruby
<record>
   status ${if record["log"].include? "error: "; 'ERROR' ; else; 'OK' end; }
</record>

Be careful around the ;(semi-colons). These might be tricky :)

SilentEntity
  • 354
  • 1
  • 4