0

We are trying to figure out if there is a way to create a stream/alert that will detect when an email address has been in a log more than X times. From what we can tell I can only count the total number of messages that match the stream.

For example, we would want to alert on a field called 'mailaddress' if the value has been the same more than 10 times in the past minute. Anyone have ideas on how to do this?

tweeks200
  • 351
  • 1
  • 3
  • 11

1 Answers1

2

There are a number of tools available that are designed for scanning logs. fail2ban is one of the these. You would need to setup an expression to match and configure the appropriate action. This could include temporarily blacklisting the user on the firewall.

You may catch poorly configured servers that have initial retry times configured in seconds rather than minute or hours. Spambots are likely to change their sending addresses frequently so you may miss them.

I've seen a number of bulk mailers retry at a fast rate using a different IP on each request. The corresponding domains, tend to be consistent on the first two or three levels.

I would fix duplicates being sent by fixing the mailing list data. You will have duplicates from temporary rejects that should be retried by your mail server. Use a reasonable initial retry like 1 hour and monitor your queue for entries that have been in the queue for a while. The domain part of an email address is always case insensitive, and the left site is almost always case insensitive.

Most mail servers will eliminate duplicate addresses on the same message. However, this doesn't help if messages are individualized.

My server will defer delivery for over an hour for every RFC violation I detect. This includes rDNS, ELHO name matching DNS, SPF and others. There are other reasons that a message will be delayed. The reason delivery acceptance was deferred should be logged.

BillThor
  • 27,737
  • 3
  • 37
  • 69
  • What I'm trying to do is find outbound emails to let us know if our systems are sending duplicates. Fail2ban would work but we were really hoping to do this through graylog since we already use it for centralized logging and alerting. – tweeks200 Jul 08 '16 at 10:59
  • @tweeks200 I've added some additional information. Checking the logs is late for checking duplicate addresses. I've taken a quick look at the graylog documentation, and it doesn't seem to be suitable for what you are doing. fail2ban can generate log messages suitable for your purposes. If they are processed by graylog, you can use graylog to search for them. – BillThor Jul 08 '16 at 13:41
  • thanks, after some further research this looks like it will work for us. – tweeks200 Jul 12 '16 at 21:48