I was going through filebeat documentation where it has promising feature to process the input logevents- https://www.elastic.co/guide/en/beats/filebeat/current/configuration-processors.html with which I should be able to mutate my logevetns.
The supported processors are:
- add_cloud_metadata
- decode_json_fields
- drop_event drop_fields
- include_fields
Unfortunately, it does not mention how to add custom Processor so that I can mutate the logevents the way I want. Say I want to process the logs in format,
timestamp a:1,b:2,c:100
to
{ "timestamp" : timestamp, "message" : { "a": 1, "b" : 2, "c": 100}}
Logstash could be one solution, but its simply an overhead to send from filebeat to logstash and then to elasticsearch.
