1

Today we realized that all of our emails being sent to email addresses of the type <username>@corporatedomain.com were getting rejected. I don't know who is hosting the email for corporatedomain.com. I don't think that should matter. Anyway, the error we were getting was 

Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the server for the recipient domain example.com by mail.example.com. [yyy.yyy.yyy.230].

The error that the other server returned was:
550 5.7.1 Command rejected
  • We have a Google Apps account for mydomain.com. The DNS for mydomain.com had the SPF record v=spf1 mx include:spf.mailjet.com -all.
  • We are using the Google Apps SMTP server to send transactional emails from with my app. We am using the mailjet account to send bulk marketing emails.
  • Since the mail for this domain is handled by Google Apps, the MX records all point to the default Google app ones.
  • My VPS has two public ip addresses. I don't have an SMTP server installed on my VPS. I send emails from within my VPS using the Google Apps SMTP server ONLY.
  • Only the emails sent via the Google App SMTP server were getting blocked. The Mailjet ones delivered fine.

Given the above usage pattern & VPS environment setup, I assumed the SPF would checkout properly (the MX records point to Google and I am using their SMTP servers to send email).

Googling the error really didn't lead to anything. Eventually I tried carpet bombing the issue by including whatever I could think of in the SPF. And now it works. Cool! The spf entry now reads

v=spf1 include:_spf.google.com include:spf.mailjet.com ip4:xxx.xxx.xxx.143 ip4:xxx.xxx.xxx.144 mx ~all

But I am not sure WHY it works. I really DONT want to do a trial & error to find the bare minimum settings that work for the recipients domain. There must be a spec for these things, right? After all, all emails were/are getting delivered to all users (not with emails belonging to corporatedomain.com).

I have a suspicion that in the current SPF entry, the ip4 mechanisms as well as the mx one are not needed. Google documentation recommends using include:_spf.google.com instead of mx. Also, this one ends with less restrictive ~all. The old SPF ended with the highly restrictive -all. Can someone confirm what the ideal SPF record should be for my use case?

Side note: I have now configured DKIM authentication for mydomain.com. I don't know if I need to setup Reverse Dns. I don't think I do, as I am not "sending" the email, the Google Apps SMTP server is. Please correct me if I am wrong.

Amith George
  • 145
  • 1
  • 6

1 Answers1

3

The include:_spf.google.com you eventually added is likely the critical bit here. It flags Google's servers as being legitimate senders of mail on your behalf.

ceejayoz
  • 32,910
  • 7
  • 82
  • 106
  • That's what even I think. But why didn't the `MX` mechanism work? The spec says it should allow all servers handling the domain's email. – Amith George Nov 05 '13 at 17:19
  • 1
    MX is for incoming mail. Google may well use different IPs for *outgoing* mail that aren't the same IPs as those MX records. As such, Google tells you to use their SPF record, which always includes their various e-mail sending netblocks correctly. – ceejayoz Nov 05 '13 at 17:21
  • If MX records are meant to be incoming only, then that would explain things a bit. How about the `~all` at the end? Google recommends it. Doesn't that "reduce" the effectiveness of the SPF entry – Amith George Nov 05 '13 at 17:46
  • 2
    They recommend it instead of `-all` because they're catering to a user base that includes people who have no idea what SPF records do. By suggesting `~all` they don't get complaints from people who wind up causing their mail sent from other servers to be rejected. If set up correctly, a `-all` record is much better. – ceejayoz Nov 05 '13 at 17:47
  • Just to clarify, if only using Gmail smtp, then I can use `-all`. There isn't a chance that the ip's returned by `_spf.google.com` might not contain the ip of the smtp server used. I got the impression that they dynamically create servers and associate ip's, so some might get left out for a very short duration. – Amith George Nov 05 '13 at 17:50
  • There shouldn't be any left out. If they purchase a netblock it's going to go into the SPF record long before they start using it for actual sends. Yes, `-all` is fine if you're only sending from Gmail (or if you've correctly added the SPF stuff for the other services too). – ceejayoz Nov 05 '13 at 17:52
  • Ok. One last thing. My VPS's ip doesn't need to be in that SPF entry, right? I don't host an SMTP server. It's just that googling the original error message led me to believe I need to include it and set up a Reverse DNS entry and all that. I may have misread that. Could you confirm I don't need to add the vps ip addresses? – Amith George Nov 05 '13 at 17:57
  • I'd personally just add it in case you ever elect to send e-mail from it. – ceejayoz Nov 05 '13 at 18:03