ElastAlert Not working

Question

I am trying to make a rule on elastAlert.

Config.yaml

rules_folder: example_rules


run_every:
  minutes: 1


buffer_time:
  minutes: 1


es_host: localhost


es_port: 9200


writeback_index: elastalert_status

alert_time_limit:
  days: 2

example_rules/example_frequency.yaml:

 name: Example rule


 type: frequency


 index: sample



 num_events: 1


 timeframe:
    hours: 4 


 filter:
 - term:
     message: "hi"


 alert:
 - "email"


 email:
 - "abc@example.com"

When I do :

GET sample/_search?q=*

I get:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 4,
    "max_score": 1,
    "hits": [
      {
        "_index": "sample",
        "_type": "blog",
        "_id": "2",
        "_score": 1,
        "_source": {
          "@timestamp": "2016-05-27T17:32:00",
          "message": "hi"
        }
      },
      {
        "_index": "sample",
        "_type": "blog",
        "_id": "4",
        "_score": 1,
        "_source": {
          "@timestamp": "2016-05-27T12:15:00",
          "message": "hi"
        }
      },
      {
        "_index": "sample",
        "_type": "blog",
        "_id": "1",
        "_score": 1,
        "_source": {
          "@timestamp": "2016-05-27T17:25:00",
          "message": "hi"
        }
      },
      {
        "_index": "sample",
        "_type": "blog",
        "_id": "3",
        "_score": 1,
        "_source": {
          "@timestamp": "2016-05-27T17:45:00",
          "message": "hi"
        }
      }
    ]
  }
}

But when I do python -m elastalert.elastalert --verbose --rule example_frequency.yaml , I get this :

    INFO:elastalert:Starting up
    INFO:elastalert:Queried rule Example rule from 2016-05-27 17:43 IST to 2016-05-27 17:44 IST: 0 hits
    INFO:elastalert:Ran Example rule from 2016-05-27 17:43 IST to 2016-05-27 17:44 IST: 0 query hits, 0 matches, 0 alerts sent
    INFO:elastalert:Sleeping for 59 seconds
   INFO:elastalert:Queried rule Example rule from 2016-05-27 17:44 IST to 2016-05-27 17:45 IST: 0 hits
    INFO:elastalert:Ran Example rule from 2016-05-27 17:44 IST to 2016-05-27 17:45 IST: 0 query hits, 0 matches, 0 alerts sent
    INFO:elastalert:Sleeping for 59 seconds
  INFO:elastalert:Queried rule Example rule from 2016-05-27 17:45 IST to 2016-05-27 17:46 IST: 0 hits
    INFO:elastalert:Ran Example rule from 2016-05-27 17:45 IST to 2016-05-27 17:46 IST: 0 query hits, 0 matches, 0 alerts sent
    INFO:elastalert:Sleeping for 59 seconds
  INFO:elastalert:Queried rule Example rule from 2016-05-27 17:46 IST to 2016-05-27 17:47 IST: 0 hits
    INFO:elastalert:Ran Example rule from 2016-05-27 17:46 IST to 2016-05-27 17:47 IST: 0 query hits, 0 matches, 0 alerts sent
    INFO:elastalert:Sleeping for 59 seconds

Why is it not working? It is showing hit queries to be 0. But why I don't understand.

The timestamps in your documents are in UTC time, i.e. 5.5 hours behind IST. You should store your timestamps as `2016-05-27T12:25:00` and it should work. — Val, May 27 '16 at 12:32
@Val : But I have also inserted UTC ones -- see the document with id 4. It should atleast return me that, Shouldn't it? — Priyansh Goel, May 27 '16 at 13:44
Yes, but it was not in the checked interval, i.e. 12:43 to 12:47 UTC — Val, May 27 '16 at 13:51
@Val : I did that and it worked. But I am still not getting emails. THough it shows something like this : `INFO:elastalert:Queried rule Example rule from 2016-05-27 19:41 IST to 2016-05-27 19:42 IST: 1 hits INFO:elastalert:Sent email to ['abc@gmail.com'] INFO:elastalert:Ran Example rule from 2016-05-27 19:41 IST to 2016-05-27 19:42 IST: 1 query hits, 1 matches, 1 alerts sent INFO:elastalert:Sleeping for 51 seconds` — Priyansh Goel, May 27 '16 at 14:19
@Val abc has been written here. I have written a valid email id there. — Priyansh Goel, May 27 '16 at 14:22
Then there must be a problem along the way, i.e. with the SMTP server, but as far as elastalert goes, it's working, judging by the logs I see. — Val, May 27 '16 at 14:23
http://elastalert.readthedocs.io/en/latest/ruletypes.html#email — Val, May 27 '16 at 14:24
@Val : I don't get this. Then what does timeframe mean? As per docs, it should look for all the documents within 4 hrs? — Priyansh Goel, May 30 '16 at 06:00
[According to the docs](http://elastalert.readthedocs.io/en/latest/ruletypes.html#frequency), the `timeframe` is the number of hours during which `num_events` must occur in order for the alert to be triggered. In your case since you have `num_events: 1` it will always be the case, but since you're only checking 1 minute, the documents are not selected. — Val, May 30 '16 at 06:08
@Val : So, for example : if it had executed at 11:00 earlier , then if run_every is 1 min then , it gets executed at 11:01 and look for all the documents between 11 and 11:01 only. But it will send alert only after every 4 hrs(which is timeframe here) and that too if the number of hits >= num_events. Is my understanding correct? — Priyansh Goel, May 30 '16 at 06:45
Yes. Did you fix your STMP server issue? Given your log, it seemed that an alert email was properly triggered and an email sent. — Val, May 30 '16 at 09:01
@Val : Yes I did get a mail a bit later. Suppose my num_events is increased to 50 and run_every is changed to 10 minutes. Now it will run every 10 mins and suppose generates hits always > 50. So, by the end of 4 hour, He will have 24 alerts that need to be sent. Will it send all 24 alerts one by one? Or will it send only one mail for all the 24 ? Or will it send the alert for the last executed hit only? I am sorry that I am carrying in that long but I am unable to understand the way it works. — Priyansh Goel, May 30 '16 at 10:08
For that you have the [realert](http://elastalert.readthedocs.io/en/latest/ruletypes.html#realert) settings which you can tune in order to get the desired behavior. It'd help a lot if you go through the documentation and run the examples in order to fully grasp this product ;-) — Val, May 31 '16 at 03:23

score 3 · Accepted Answer · answered Jul 19 '16 at 07:11

Using the "--es_debug_trace" can help like in the example given below

python -m elastalert.elastalert --verbose --rule example_frequency.yaml --es_debug_trace /opt/elastalert/runtime.log

This can help you get a look at actual cURL command being fired to get the number of hits. Here you can look at the date/time range being used to search for your filter/queries/matches.

In your case the problem was the date(IST and UTC) as mentioned by @Val in comments.

score 0 · Answer 2 · answered May 27 '16 at 13:55

You need to configure the timestamp in your rule (example_rules/example_frequency.yaml)

timestamp_field: "@timestamp"

and possibely :
timestamp_type
timestamp_format
-> Documentation

in addition to that, in your case, you will have the best performance with these configurations:
use_count_query: true
doc_type: blog
-> Documentation

ElastAlert Not working

2 Answers2