Prevent SPAM by limiting SMTP connections using iptables

Question

We operate a private VPN service which has been recently abused by spammers and we would like to be able to limit the number of SMTP connections per minute/hour to make it ineffective for spammers whilst still functional for normal use. I did my homework and came up with the following, but we are still receiving spam reports. Any better ideas or is this approach flawed in some way?

# if it has more than 60 connections in a 120 seconds interval: DROP
iptables -A Limit_SMTP -m recent -p tcp --update --name ovpn_smtp --seconds 120 \
  --hitcount 30 -j REJECT --reject-with tcp-reset

# otherwise: allow
iptables -A Limit_SMTP -m recent --set --name ovpn_smtp -j ACCEPT

Answer 1

The comment in your snippet doesn't match the code: --hitcount 30 is limiting this to 30 "hits"... and unless you're using --state NEW somewhere, that's going to be 30 packets, not 30 connections. Without knowing how you're getting to the "Limit_SMTP" chain, it's impossible to say this is even being used.

Fundamentally, though, the issue is that once I connect to an SMTP server, I can send as many emails as I'd like before I disconnect, unless the server forces me off itself.

Depending on what exactly it is you're trying to block, you might consider joining the increasing number of ISPs blocking port 25 and forcing their clients to use port 587. The upside of this is that people will only be able to connect to and send mail from mailservers they have an account on (or badly misconfigured open relays), making any spam they send become that mailserver's problem. The downside of this is that they'll only be able to connect to and send mail from mailservers they have an account on and has been set up to accept connections on port 587. That means they can't run their own mailserver "inside" your network unless it's configured to take all their mail and send it to a mailserver "outside" of your network (aka smarthost) which then sends it to the destination servers.