2

So I have a certain query running in (Yelp's) Elastalert and I am trying to filter out logs containing one of several keywords. If I use the any rule type, I get a set of 30 matches to the certain query I have. When I change the ruletype to whitelist:

type: whitelist
compare_key: message
ignore_null: true
whitelist: ["exclude_strings"...]

I still get the same 30 matches, even when I know the message field contains the listed strings. I've also tried changing the compare key or the strings, using strings that exactly match the entire field, I've changed the formatting to

whitelist:
- "string"
...

and nothing has made a difference. The same thing happens also with the blacklist type.

What am I missing?

TallChuck
  • 1,725
  • 11
  • 28

1 Answers1

2

After further testing, it turns out that either of the above formats will work correctly. The reason I thought it was not working is that I was looking at the hits term in the Elastalert status. Instead I should have been looking at the matches term. The search returned the same number of hits because the query was the same each time, but it seems that the matches term comes, not from ElasticSearch, but from Elastalert itself.

That is, Elastalert sends the full query to ElasticSearch, and then does the filtering on the returned data based on the whitelist terms. hits will be the same every time, but matches depends on the whitelist. If you set realert to zero, you will see that the number of alerts generated is the same as the number of matches.

TallChuck
  • 1,725
  • 11
  • 28