SPF is a foundational component of email deliverability yet proper implementation has eluded a significant portion of the Internet for nearly 15 years. We believe that making SPF easier to implement correctly will allow us all to enjoy a safer Internet. That’s why we’re launching Universal SPF, an in-line upgrade to traditional SPF. As of today, every known server and mail gateway that was capable of traditional SPF now supports Universal SPF. Say goodbye to permerrors, excessive DNS lookups, SPF record flattening, problematic ESP verification, and a host of other challenges.
We'd like to thank the many email deliverability experts who helped test Universal SPF over the past few weeks.
Seeing is believing
Let’s work through a few examples together so you can see Universal SPF’s “magic string” fix current policy problems and protects against future issues.
Here's a common but broken small business policy: Bluehost plus Office 365. Here is the seemingly reasonable SPF policy:
v=spf1 include:bluehost.com include:spf.protection.outlook.com -all
Traditional SPF limits us to 10 DNS lookup terms. At first, it might seem the above policy has only 2 lookups but an advanced SPF record checker reveals all 14 lookups. In reality, this tiny policy exceeds the DNS lookup limit by 40%.
As the screenshot shows, SPF Compression is one way to fix our policy, but it’s a sophisticated policy management service beyond the scope of this article. We’re here to explore Universal SPF, so let’s dig in.
We’ll evaluate our test domain, mailproblems.com, using a tool available in your macOS terminal. What happens when we send mail from 52.100.0.0, one of the IPs that our overflowing policy tried to authorize?
% spfquery --id me@mailproblems.com --ip 52.100.0.0
permerror
Until now, companies without budgets for SPF Compression have been left trying to follow all manner of incorrect or impractical advice, such as rewriting their policy or, worse, manually flattening the policy from DNS lookups to raw IP addresses. Such techniques are infamous for drifting out of date as soon as they’re published to DNS. Remember, if you flatten your own policy, your mail may fail SPF every time your vendor changes IP addresses.
We believe you have better things to do than constantly monitor and update your SPF policy, and that’s why we’re giving you Universal SPF. Let’s try it out together.
Fix my SPF
Upgrading to Universal SPF just requires replacing the front of the record with
v=spf1 include:UniversalSPF.org -include:x.UniversalSPF.org
That's it. That's the "magic string" we'll add to our old SPF record in Cloudflare.
Note that our TTL time is set to the minimum of 2 minutes, so we shouldn’t have to wait long for our change to propagate. We’ll dig the policy a few times until our local machine has the new version.
% dig txt mailproblems.com +short
"v=spf1 include:UniversalSPF.org -include:x.UniversalSPF.org include:bluehost.com include:spf.protection.outlook.com -all"
Let’s re-evaluate our SPF record after upgrading to Universal SPF. Tap the up arrow twice to recall spfquery and execute it again.
% spfquery --id me@mailproblems.com --ip 52.100.0.0
pass
Our desired message handling of “pass” shows that mail from this IP is authorized for our domain! As an inline upgrade to legacy SPF, Universal SPF has interpreted our old policy and then deactivated it so that the large number of DNS lookups no longer causes a policy failure.
Hang on, what just happened?
- Acting like a mail gateway, your computer recognized the domain’s Universal SPF policy and connected to the closest of Fraudmarc’s network edge nodes. We have about 100 locations spread strategically across the planet to ensure high availability and low latency.
- Fraudmarc’s Universal SPF edge node connected through our private backhaul network to the closest regional cluster. Clusters are presently located in Asia, Europe, and USA. Third-party testing indicates our global average warm query time is already below our 100ms target. Free, error-free & faster than traditional SPF.
- A test account was created for your domain. It’ll remain active for a little while so you can gain confidence in your new policy. To continue leveraging Universal SPF, you’ll need to send us one activation email from the domain to be served.
- Universal SPF imported your legacy SPF policy and then inactivated it so that DNS lookups, loops, or dupes will no longer break your record.
- The inactivated legacy policy remains visible in DNS so that your current and future email service providers can easily verify that you’ve added them to your SPF.
- Universal SPF responded that the queried IP is a valid sender for your domain. This is correct, as it's presently in Microsoft Office 365's include policy.
Go on, do it again
Let’s see if Universal SPF can overcome another class of SPF policy mistakes: loops.
We’ve recursively called our own domain, causing a loop. This is surprisingly common. It’s not usually front and center like this but rather buried inside “cleverly” nested records, where it’s harder to identify and eliminate.
As before, we’ll first dig to ensure our DNS has updated.
% dig txt mailproblems.com +short
"v=spf1 include:mailproblems.com include:bluehost.com include:spf.protection.outlook.com -all"
Yep, our looping policy is ready for evaluation.
% spfquery --id me@mailproblems.com --ip 52.100.0.0
permerror
The result of permerror confirms our SPF is again broken. Let's see what happens when we upgrade to Universal SPF. First we confirm DNS changes with dig:
% dig txt mailproblems.com +short
"v=spf1 include:UniversalSPF.org -include:x.UniversalSPF.org include:mailproblems.com include:bluehost.com include:spf.protection.outlook.com -all"
Now evaluate the Universal SPF policy with this infinite loop.
% spfquery --id me@mailproblems.com --ip 52.100.0.0
pass
Again, again!
Universal SPF has no problem with loops! Next up: dupes (duplicates). Let’s repeat a bunch of the terms.
We added another copy of Bluehost and our own domain, so our little policy now has dupes, loops, and dupes of loops. Not to worry: our domain is protected by Universal SPF.
% spfquery --id me@mailproblems.com --ip 52.100.0.0
pass
Universal SPF works, verifying the sending IP despite a garbage legacy SPF policy.
It can pass, but can it fail?
We've confirmed that Universal SPF enables good mail to pass. Let's also confirm that Universal SPF blocks all illegitimate mail sources using your “-all” term.
Let’s test with a randomly chosen IP that's not included in our policy: 1.2.3.4
% spfquery --id me@mailproblems.com --ip 1.2.3.4
fail
Perfect! This shows that bad mail is blocked by our Universal SPF policy. Contrast this with legacy SPF, which simply errors out from a broken record, neither passing good mail nor failing bad mail. I’ll remove Universal SPF’s magic string to show you.
Ensure DNS has propagated:
% dig txt mailproblems.com +short
"v=spf1 include:mailproblems.com include:bluehost.com include:bluehost.com include:mailproblems.com include:spf.protection.outlook.com -all"
Now evaluate the policy.
% spfquery --id me@mailproblems.com --ip 1.2.3.4
permerror
Permerror means that without Universal SPF, the policy is broken and cannot successfully fail the bad source of mail. Yikes!
Conclusion
We’ve observed that Universal SPF just works where legacy SPF falls short, unable to pass good mail or fail bad mail. Upgrading is fast, free, and already supported by every legacy-SPF-capable mail gateway across the Internet. Give it a whirl, and remember to email us to fully activate your domain after testing. Upgrade to Universal SPF.
Next time, we’ll pick on a few big companies that publish broken SPF records and demonstrate how Universal SPF would effortlessly solve their email authentication and deliverability problems. Tell us if there’s a particularly epic SPF failure you’d like us to cover.
If you're obsessed with inventing elegant solutions to impossible problems, let's be friends and maybe work together.
