SPF Include Statements Still Not Passing

Question

Like many others, I have navigated the SPF/DKIM/DMARC world with some confusion. About 4 weeks ago or so I finished setting everything (SPF/DKIM/DMARC) up correctly for a GoDaddy-hosted domain that uses Google's mailservers.

I set the _dmarc TXT record to take zero action with p=none and I used Postmark to monitor the results to see what was passing and failing over a week.

After a week or so I looked at the Postmark results and inserted the include: statements for the domains that I wanted to pass, but weren't. Then I waited another week to see the results. However, the results showed that the domains still weren't passing SPF or DKIM. Below is the SPF record, I've redacted parts of it that are revealing, but two of the domains are legit and still aren't passing.

v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all

Do I need to use the actual IP addresses in the include statements instead of the domains? Postmark lists these as well so that would be easy if so.

Answer 1

No, you shouldn't copy their IPs in there because they are subject to change, especially Google's.

If it's failing, presumably you have some results (usually in message headers) that tell you exactly which IP is failing, and you can track it down manually though those includes, do a reverse lookup on it, etc.

However, you're also using GoDaddy, which is mostly guaranteed not to work as they either block outbound SMTP or route it through their own servers, so you're very unlikely to get an SPF pass.

Answer 2

The issue was with SPF DNS lookup limits. I had no idea this was a thing and I'm amazed that this isn't mentioned anywhere on the documentation (whether that's Google's official documentation or otherwise) on setting up SPF/DKIM/DMARC, and didn't come up in Googling of this issue. This limit is designed to prevent denial of service attacks and infinite DNS loops.

For anyone else who sees this post

v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all

This SPF record actually has almost 15 DNS lookups, and the limit is 10 per domain. You can find out how many SPF DNS lookups your domain has with a service like AutoSPF or Easy DMARC

The solution, once you see your total DNS lookups, comes in four options:

1. Create subdomains and use those to diversify the records. For example using "email@business.mydomain.com" as the email for freshemail.io. Then on the SPF record for that subdomain, you would only have v=spf1 include:freshemail.io resulting in less than 10 DNS lookups for that domain.

2. As @Synchro mentioned, you don't want to use IPs because those can very well change, but the concept of using IPs instead of the domain names does essentially work because an IP address doesn't cost a DNS lookup. Check with the support/engineering of whatever service you're using, it's possible that they have an IP (or an IP range) that doesn't change often. You might be able to bring your DNS lookups under ten using this.

   Note that Google takes up about 3 DNS Lookups, and you'll probably want to leave that one as the _spf.google.com value

   Note that every SPF record also has a 255 character limit, so if you're using only IPs you'll need to break that up into a lot of SPF records probably

3. Use an SPF flattening or compressing service like AutoSPF. Essentially, these services employ method #2, but do some backend work every few hours to check and update the IP addresses associated with the domains. Then they provide you with a "compressed" record like v=spf1 include:_6359384.autospf.com ~all that references all of your records and results in far fewer DNS lookups.

4. Create your own method that acts kind of like #2 and #3, using GoDaddy's API and brew up something that performs updated lookups on a schedule/job and updates separate SPF records including all of the IPs.