Filebeat for gz files

Question

Does filebeat support ingesting from gz files? If I unzip the files before ingesting, they grow in size very big.

I did a search on the Elasticsearch forum, but could not find anything useful.

check this link https://discuss.elastic.co/t/filebeat-to-stream-gz-logs-to-es-ls/121081 — Zee, Apr 19 '18 at 13:30

score 3 · Answer 1 · answered May 22 '20 at 12:22

3

Filebeat doesn't support gz files right now. But there is an ongoing request for that, you can track the progress here.

https://github.com/elastic/beats/issues/637

answered May 22 '20 at 12:22

Deepan

119
1
2
8

score 0 · Answer 2 · answered Apr 19 '18 at 20:27

Filebeat will not be able to properly consume contents inside GZIP files. It will read it but the output will be unreadable i.e. UTF-8 encoded garbage. The Elastic team suggest such files get excluded, see below.

exclude_files: ['\.gz$']

For further details please check the following link: discuss.elastic.co/t/filebeat-to-stream-gz-logs-to-es-ls/121081

Filebeat for gz files

2 Answers2