Does filebeat add metadata while sending logs to elasticsearch?

Question

I'm trying to send systemd logs to elasticsearch via filebeat. I see the following metadata field in elasticsearch for one of the log entries.

"meta" : {
        "cloud" : {
          "instance_id" : "xxxxxx",
          "machine_type" : "xxxxxx",
          "instance_name" : "xxxx",
          "availability_zone" : "xxxxxx",
          "provider" : "xxxxx"
        }
      },

I was wondering if the above metadata field visible is added by filebeat or its a field picked up by systemd and just forwarded to elasticsearch as is by filebeat? Is there a way to check that? Or does filebeat by default add metadata to logs that it forwards?

score 1 · Accepted Answer · answered Nov 14 '19 at 00:55

Yes, it does add metadata fields.

Look into your filebeat.yml, you will probably see the two lines below:

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

The host and cloud metadata are enabled per default, if you don't want those fields you just need to comment those lines.

Does filebeat add metadata while sending logs to elasticsearch?

1 Answers1