While using kafka input, I want to output only when json data contains a specific string.
I tried setting "include_lines" in filebeat.yml, but it was not filtered properly.
When the filebit.yml setting is as follows and data-set1 and 2 are input, not only data-set1 but also data-set2 are output.
I expected only data-set 1 to be output, but it wasn't.
What did I make a mistake?
part of the filebeat.yml
filebeat.inputs: - type: kafka hosts: - qa-parkbae-01.hanpda.com:9092, - qa-parkbae-02.hanpda.com:9092, - qa-parkbae-03.hanpda.com:9092 topics: ["parkbae-test-topic1"] group_id: "test123" ssl.enabled: false include_lines: ['\"event\":\"basket\"']
input data-set1 :
{"id":"parkbae","event":"basket","data":"test1"}input data-set2 :
{"id":"parkbae","event":"ball","data":"test2"}