Mail rejected due to SMTP Spam Blocking: _SPF (Fail)
Problem reported by CCC - 10/16/2014 at 10:34 AM
We have recently received reports of messages being intermittently rejected due to SPF failure.  We are running the latest build of smartermail 12.x
Our server is configured to block SMTP connections that fail SPF checks.
We have checked the source IP against their SPF record using this site and the SPF record passes.
Most of these appear to be coming from *.outbound.protection.outlook.com
In order to rule out the local DNS server as the culprit, we set smartermail to use a DNS server on the same LAN instead of the local DNS server.  No change in behaviour.
Is anyone else seeing this issue?  At this point, I believe our only recourse is to disable blocking SMTP connections that fail SPF, but that seems like it would just allow more junk mail to be filtered.  I'd much rather block SPF failures, but these seem to be legitimate.

7 Replies

Reply to Thread
Joe Wolf Replied
First off you should never SMTP Block on any ONE test.  No tests are 100% accurate and if you're going to SMTP Block you should require the message to fail at least two tests.
Secondly, if you SMTP Block on SPF failure you're going to block a LOT of valid messages that were simply forwarded thru a server that doesn't support SRS.
If you're going to do ANY SMTP Blocking make sure you disable the SmarterMail DNS cache and use your own DNS resolver that doesn't cache, or lets you set a maximum TTL of 1 minute or so.  DNS queries use very little data and there's no reason to risk blocking messages due to stale records.
A lot of people use registrar DNS services.  Most of them use a default TTL of 86,400 seconds (one day).  So if your resolver honors TTL it will use the existing cached record until TTL expires.  So even if the real DNS data was changed or corrected you're blocking based off stale value.  
CCC Replied
Hi Joe,

Thanks for the followup.

Will try lowering the weight we assign to hard SPF failures and see if that helps. Although it seems like my next forum post will be "mail periodically ending up in junk mail folder due to intermittent SPF failure" :)

We figured if the customer is telling the world to reject mail if it doesn't come from their defined mail servers (using -all in their SPF record), we should honor that request and we set the spam weight to blocking levels. We don't block on soft fail (~all), we just assign it a lower spam weight.

We have DNS caching disabled (under protocol > SMTP out), are there additional settings for DNS caching elsewhere within SmarterMail?

Next step will be to install ISC Bind locally with no caching.

CCC Replied
Still digging into this issue.  We have recently lowered RDNS and SPF weights to non-blocking levels to get the mail flowing (albeit to the junk mail folder) but in looking at the SMTP logs from the day before yesterday there still appears to be something strange going on and Outlook.com appears to be "the canary in the coal mine".
Searching SMTP logfiles with the following regular expression returned a list of all transactions that were blocked solely due to SPF and reverse DNS failure
More then 53% of the IP addresses returned by that query ended with *.protection.outlook.com
Just under 25% spread across individual domain names - only 1 of which i have heard of before, the rest look very spammy (based on their name)
Just under 22% had no reverse DNS
Granted, I am checking these more then 24 hours after they happened, but it seems suspect that outlook.com appears in there more then 50% of the time, and no other domain repeats more then twice.
The reports we have gotten from our customers in recent history have consistently involved mail coming from an outlook.com mail server.
Just wondering if anyone out there is seeing similar issues with mail from outlook.com and/or if they would be willing to dig into their SMTP logs to see if they are seeing similar behavior.
CCC Replied
Interesting reply I found in a post in another forum
Outlook.com's SPF records spiral down so deep into DNS lookups that you cannot add [m?]any more include: directives without violating SPF's RFC-defined limit. Most SPF validators don't strictly enforce this, but some do and can cause deliverability issues at random.
CCC Replied
As of November 2014, SmarterMail does not differentiate between a typical SPF failure and an SPF failure due to SPF processing limits. The ability to log those failures differently would make it easier to troubleshoot and go after the offending senders. http://portal.smartertools.com/community/a696/does-smartermail-strictly-enforce-spf-processing-limits-as-defined-in-rfc-4408.aspx
Steve Reid Replied
Smartermail needs better SPF failure logging and handling.
David Finley Replied
We are using SM 16 and for the first time we see this issue. We have SPF failures on domains with no published SPF records? Weird!

Reply to Thread