0

I'm new to fluentd and using it to ingest data to elasticsearch. I have a field called request_time which has value : 110820120501 as a number of the format [ddmmyyHHMMSS]. How can I convert this as a date field.

This is what I've written but it doesn't work. Any help is appreciated.

Example request_time value : 100820015642

    <filter>
        @type record_transformer
        enable_ruby true
        <record>
        request_time ${ require 'date'; DateTime.strptime('request_time', '%d%m%y%H%M%S')}
        </record>
    </filter>

I have tried the below as well:

    <filter>
        @type record_transformer
        enable_ruby true
        <record>
            time ${require 'date'; DateTime.parse('request_time').strftime('%d%m%y%H%M%S')}
        </record>
    </filter>

How can I store request_time as a date field coverted to a value as eg. "request_time": "2020-08-11T11:24:23.000+0100" ?

Preyas
  • 244
  • 2
  • 7
  • What do you mean by 'does not work?' parsing the example request time value produces an expected results. – benjessop Aug 11 '20 at 13:49
  • I get this error when I run the td-agent with the first code snippet. ``` fluent/log.rb:306:debug: map:{"%Q[request_time]"=>" require 'date'; DateTime.strptime('request_time', '%d%m%y%H%M%S')"} ``` and gives the value as : "request_time"=>"100820022014"}. But I'm looking for the output to be like : "2020-08-10T02:20:14.000+0100" – Preyas Aug 11 '20 at 13:58
  • Got it working using the below : ``` time "${require 'date'; DateTime.strptime(record['request_time'],'%d%m%y%H%M%S')}" ``` – Preyas Aug 23 '20 at 15:19

1 Answers1

0

Pleas try the below snippet. I have given milli second precision, but feel free to cut it down to your need.

<filter>
    @type record_transformer
    enable_ruby true
    <record>
    date ${Time.at(record['request_time'].to_i/1000, record['request_time'].to_i%1000*1000).utc.strftime('%Y-%m-%dT%H:%M:%S.%LZ')}
    </record>
</filter>
Arun M
  • 1