2
Incorrect SPF fails
Problem reported by Gary P - 3/11/2021 at 2:51 AM
Submitted
We have had a ticket open with ST since Jan 6. Scenario is email from office 365 users fails SPF check incorrectly. 

Example: 

Received: from GBR01-LO2-obe.outbound.protection.outlook.com (mail-eopbgr100058.outbound.protection.outlook.com [40.107.10.58]) by xxx.xxx.com with SMTP

This FAILS SPF check, even though it is valid.

As a work around we added Office 365 sending IP blocks to IP Bypass list in this case the 40.107.0.0/16 with SPAM check bypass ON.

Outcome -  all email from Office 365 still gets marked as SPAM

X-SmarterMail-Spam: Reverse DNS Lookup [ReverseFailed]: 5, Null Sender: 0, SPF [Fail]: 10, DK [None]: 0
X-SmarterMail-TotalSpamWeight: 15

Anyone got same issue and have a fix ?

8 Replies

Reply to Thread
0
Douglas Foster Replied
We use the Python module pyspf, which was written by the people who created the SPF spec.  We made some modifications to make it return results as a bit-mapped exit status for compatibility with Declude.  We just added some more logic to distinguish NONE from non-existent domain.   Returns all SPF result codes, not just FAIL.   Declude filters let YOU decide what to do with each result and allow you to define exceptions for incorrect policies.

SM calls Declude.  Declude calls a vbscript custom filter.  Vbscript launches Python script.  Sounds ugly but works well.

Send me a message if you want a copy.
0
Sébastien Riccio Replied
If I remember correctly the spam checks bypass does not skip SPF check. Or maybe it was for Trusted senders, I don't remember exactly as we don't use the built-in SmarterMail spam checks anymore.

It seems that in your case the reverse lookups also fail which adds 5 to the spam weight:

It looks like their HELO and reverse doesn't match the IP reverse DNS. It should match.

GBR01-LO2-obe.outbound.protection.outlook.com (mail-eopbgr100058.outbound.protection.outlook.com [40.107.10.58])
# host 40.107.10.58
58.10.107.40.in-addr.arpa domain name pointer mail-eopbgr100058.outbound.protection.outlook.com.
The IP seems indeed to be in outlook.com SPF record:

But is this the case of the domain name of the sender if they use anything else than outlook.com ? Have you already checked this?

Kind regards.
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Gary P Replied
Douglas Foster   -  thanks definitely interested, but I agree sounds complicated , however reality is and i hope you agree,  this check should work without the need for this. 


But is this the case of the domain name of the sender if they use anything else than outlook.com ? Have you already checked this?
 I am unclear what you mean. Senders using Office 365 tend to use MS servers for sending email ? We are seeing this failure on almost all email that arrives from senders who use office 365. 

As for reverse DNS - this is a known issue with Office 365. 


 I don't understand the logic of having a user interface that allows admins to add IP Bypasses - with option to ByPass SPAM Checks - then ignores a SPAM check bypass.


0
Kyle Kerst Replied
Employee Post
IP Bypass will only skip spam checking and greylisting for that IP, and will apply those spam checks to the Received line preceding that server in the message envelope chain. We'd need to see the full message headers and SMTP/Delivery/Spam Checks logs to see what IP is ultimately being scanned and failed. 

I think in this case you're looking for either a whitelisting or finding the root cause behind the SPF failures. Do you use 1.1.1.1/8.8.8.8 for DNS or do you have your own servers in place? Perhaps O365 recently added to their SPF and you have a cached result in there? 
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
Sébastien Riccio Replied
Hello Gary,

I was trying to ask if the sender domain name (in Return-Path and From headers) is outlook.com or if the senders are using their own domain name (as you can use your own domain name in o365).

If they use their own domain name with o365, is the SPF of the domain name including the record for o365 ?

Kind regards.
Sébastien Riccio System & Network Admin https://swisscenter.com
1
Millennium Systems Replied
Gary,

We had the exact same issue (as well as false rDNS failures) not long ago and resolved it with Kyle's help by changing DNS and IP Bypasses. 

We changed one of our SM DNS IPs to 8.8.8.8 which showed an improvement overall but we still had occasional false failures for O365 senders. We had various IP ranges in Antispam > IP Bypasses for greylisting, after migrating these to Antispam > Greylist Filters (and the DNS change) we are no longer seeing false SPF and rDNS failures from O365

Detailed logging on spam filtering showed that when the IP is in the IP Bypasses the SPF check was looking at a previous header in the chain and failing.
This wasn't quite clear us either and could be better documented but were advised the Antispam > IP Bypasses should only be used when you deploy incoming spam gateways. Use Antispam > Greylist Filters and Security > Whitelist otherwise.
0
Sheila Micthell Replied
how can I add SPF records for my site? 
0
Douglas Foster Replied
Authorizing your own mail for SPF:

The official reference for SPF is the IETF document:   https://tools.ietf.org/html/rfc7208

Inferring from other sites that use O365, your SPF policy should have this element included:
   "include:spf.protection.outlook.com"
This will authorize outlook.com to send your mail, and protect you from risk of message rejection by other organizations.    


Testing SPF on incoming mail:

If a desired sender domain uses O365 and has not done this, you need to configure an exception for that domain (or disable SPF checking completely.)   I have never been satisfied with the exception mechanisms in SmarterMail, which is why I use Declude.   But SPF policy errors are common, and SPF FAIL is more likely to indicate a configuration error than a fraudulent impersonation.   As a result, I have found blocking based on SPF FAIL to cause unacceptable problems, so turning it off is an option.

I find the actual value of SPF testing is to combine a desired domain with SPF PASS to permit whitelisting.   The domain name tells me that it is a preferred sender and the SPF PASS tells me that the preferred sender has not be impersonated, therefore the whitelisting action is safe.   I could not find a way to do that with SmarterMail spam filtering by itself, or in a bunch of other commercial solutions, in though some were very expensive.

I have recently begun using Declude to test for non-existent SMTP domains, and that seems to be a pretty reliable indicator of spam.   But I don't think SmarterMail provides a way to detect that.

Miscellaneous on O365 server names
SPF uses a forward-confirmed HELO name when the SMTP address is null.   The HELO name is irrelevant for SPF otherwise.   My data shows that Office365 HELO names do not forward confirm to the IP address, but the Reverse DNS lookup of the IP address will forward-confirm back to the

Reply to Thread