You can set SmarterMail to accept mail that fail SPF.

We tested it and it's fully functional, even if I don't recommend it ...

Simply set Antispam --> spam checks --> SPF with the settings that better fit your policy.

These are mine (I don't accept mail that fail SPF...):

Once this is done you can customize the antispam behavior of each individual Domain (or even of each individual user) based on the SPAM values:

P.S.: normally it is right to refuse a mail that fail SPF test...

Usually when a client complains about this I give him a consultation (for a fee, not for free ...) in which I analyze the emails that have been blocked, I show the client that the emails fail the SPF test even with external tools (example MXTOOLBOX) and I explain to him well and how SPF works, showing him also wikipedia or other sites that explain what it is.

Finally, I advise him to report to those who sent him the blocked email to better check his mail servers and his DNS settings, also telling him if their managers are not able to do it, we are available to advise them.

We acquired several customers this way ...

That's very helpful. Thank you!

We were able to figure out the problem. One of my clients is a university. My related emails go to their .edu address. I have an Outlook Web app auto-forward set to go to my address that is managed by SmarterTools. 

There's something about that auto-forward feature in Outlook that flags it.

I've changed it from auto-forward to using Rules to forward messages. We'll see how that works!

Thank you for your help!

I would disagree strongly with your hosting service.   SPF is poorly corelated to unwanted mail.   Correlation is probably negative - spf fail most often means a configuration mistake by the sender and is probably traffic that you want.

To analyze the problem, you need to collect (at least)  the subject line and Message From header as well as the SPF parameters.    Then see if it is actually hostile traffic.

SM logs by themselves do not capture the Message From, so they need Declude or another tool, as well as log parsing tools, to do so.   A message journal that captures all logs would also be viable, but more difficult.

Mail tuning requires a tuning mechanism.  Flying blind is wrong.

This is way beyond my expertise, but I will say that each one we've reviewed has not been a hostile sender. To be fair, what I don't know is how many hostile senders have been stopped.

Overall, my frustration with SmarterTools is that I can specify a domain or sender as Trusted and it still either shows up in Junk or gets rejected because of something like an SPF fail. This didn't happen before the big upgrade....

I pulled some numbers from my incoming message history.  

These results are hardly an endorsement for filtering on SPF FAIL.   Negative correlation is not quite right. Unrelated is a better statistical summary, because FAIL is wrong about 50% of the time.  You do not have the resources to run down everybody else's mistake, nor do you have the ability to make them fix their mistakes.  Also note that a lot of garbage message pass SPF perfectly well.

I do not know any reason why SmarterMail would be behaving differently compared to earlier versions.   The definition of SPF FAIL has not changed.   

Note that there are possibilities between Pass and Fail when a complete SPF algorithm is used, but SmarterMail lacks that sophistication.   

These numbers have a small amount of double-counting:   

Some of the not-blocked traffic is also blocked downstream.   

These results were obtained with SmarterMail configured as an incoming gateway, calling Declude.  Then Declude has an external filter which invokes a Python script developed by people involved in the SPF standards process.   Log parsing is needed to summarize the results into a database.  (Your hosting service could do the same with no licensing cost.  They just need to contact MailsBestFriend, or myself.)

 

Hi Douglas, you're right, sometimes blocking emails because of failed SPF can be a problem, so I say you can decide which policy to use.

As far as we are concerned, however, we follow this logic:

if in the SPF record a MAIL server manager puts:

So I block the mail that FAIL spf test and accept with reserve the e-mails that SOFT FAIL the spf test.

If an email FAIL (not SOFT FAIL, the real -all FAIL) the spf test even if it actually comes from an IP that the managers of the starting domain deem valid then it is their mistake and they must correct the SPF record, it cannot be me who accepts all the emails, even those who clearly FAIL the test, because of those who are unable to manage their mail servers correctly ... 

If a manger is unsure of wich IP is valid, he ca use "~all" (SOFT FAIL) or even "+all" (all IP PASS - I don't recommend it) or "?all" (ip not listed are NEUTRAL - I don't recommend it either).

My 2 cents, everyone can agree with me or not, and therefore everyone can adopt the policy they prefer.

I will belabor this a little, because it is at the core of my frustrations with the email filtering industry.

SPF would work if all SPF records were exactly correct, and if no legitimate senders ever infringed on someone else's SPF rule.   Both are false.

If a vendor is developing a tool that uses both address and IP to check an SPF record, it would seem obvious that the system also needs a way to bypass SPF using an address and an IP.    Wow! A 2-attribute rule needs a 2-attribute exception!   

A little time spent looking at mail streams will add complexity.  The exception is not really for address and IP, it needs to be an exception for sender address and server farm.  One such farm is outlook.com, where the list of all IPs is large, unknown, and unknowable.     So the system needs the ability to define exceptions using address and server host name for complex server farms, as well as address and IP for simple situations.

But host names can be fraudulent, so we really need to know that the host name can be forward-resolved to the Source IP address before it is completely safe to use the host name to reduce filtering.

But every mail message has two potential host names, the HELO name and the ReverseDNS name, some will be verifiable and some will not.   In my dataset, as a percent of messages, I have 7% unconfirmed, 16% confirmed on HELO name only, 15% confirmed on ReverseDNS name only, and 62% confirmed on both names.

All too many email filtering products, including SmarterMail's custom rule feature, can only filter on a single attribute.   So there is no good way to override an SPF problem.   I can whitelist the IP or whitelist the Sender or whitelist the server dns names (by filtering on header=Received).   Trying to build a multi-attribute rule from a series of scores will quickly prove difficult to sustain,.

But SmarterMail is not primarily an email filtering product, so its limitations are not surprising.    What shocks is the number of specialized email filtering products that cannot do multiple-attribute rules for problems like SPF exceptions.   In many case, there is also no granularity to the exception process -- if you want to override SPF, you agree to accept everything that is whitelisted short of obvious malware attachments.

I categorize incoming mail like this:

Spam filtering has as it's goal to block 100% of group 4 and as much as possible from group 3.    It can have false positives that extend into group 2, but the system must have the ability to implement exceptions to pass everything in group 1.

The point of Mr. Kaufmans' question was that his messages from group 1 are being blocked, and the mail system administrator does not have an adequate tool for creating an exception to allow the message.    Given the limitations of his toolkit, the appropriate response is to disable SPF filtering.    Since they are unwilling to do so, Mr Kaufman should find a new mail system vendor.

Besides, if the mail system vendor is only using SmarterMail for spam filtering, the filtering is probably inadequate anyway.

I have learned a lot through these replies. Thank you.

Douglas, I think this revolves around the fact that stuff can still appear in junk despite being on the trusted sender list, though you added a lot of other useful information. If you are using both SPF and trusted senders we CANNOT let users who fail get by without getting spam checked. The malicious attack would otherwise be simple, if I can SPF FAIL and still send to you as a trusted sender (or one of your contacts) then I can send you viruses and spam. SPF FAILING is sometimes ok, but in these cases WE MUST rely on the other spam checks (which also flagged the message in Andy's case) otherwise we would be giving spammers AN OPEN INVITATION TO SEND YOU LITERALLY ANYTHING.

SPF failing can happen, us not trusting your trusted senders because they aren't a valid sender CANNOT BE CHANGED. BUT in these cases we defer to the other checks and your other checks have ALSO FLAGGED THE MESSAGE. In these cases we have no choice but to trust your spam checks and deliver the mail accordingly.

This puts us in a weird spot because there is no easy list that you can add an email address to and get a guaranteed non-spamchecked email delivered to the inbox. Again, in these cases your other spam checks should do the scoring hopefully accordingly. WE CANT TRUST SPF FAILED MESSAGES.

Why did this behavior change in this version (version 17 over 16)? Because your contacts are counted as trusted senders as well as other gal users. I'd say many people have an "admin@yourdomain.com", this potentially makes it easier for spammers to guess who is on your trusted senders list and ATTEMPT to maliciously deliver you messages with an SPF FAILED message. The "issue" this solves technically still exists in previous versions, it's just not as easy to guess who is on your trusted list.

Yes, you make an important distinction, which I missed.   I was speaking of the reasons why SPF FAIL as a unilateral block decision is problematic -- I have tried to implement it several times, and always had to roll back quickly because of "category 1" messages being blocked.

Sender authentication in all of its forms, including SPF, is an important tool when deciding to give reduced filtering to a source.   Trust requires more than the sender address, because addresses are easily spoofed.   SPF is one way of doing so.  Requiring SPF PASS would be an even better mechanism than requiring NOT SPF FAIL.   But there are situations where the trust needs to be defined by another mechanism, such as forward-confirmed DNS, exactly because the sender's SPF is messed up.

Nonetheless, one must also consider the possibility that a trusted sender has an infected machine or a compromised account.   Deciding whether this particular message is a threat or not requires more study than I have given it.   But if the answer is that it is "category 1", then an exception is needed for it.   Too often, the policy becomes constrained by what the exception system can represent.    

I currently support commercial spam filters from 3 different companies, and none of those three have multi-factor exceptions.   I have been using all of those other products long enough for them to "mature", but the feature is not even on the planning horizon.   

So then I spent six months looking at commercial alternatives, and was equally frustrated.   I came to the conclusion that email filter vendors did not understand the email filter problem that they purported to solve.   Some of the most expensive products were ruled out on price, but their capabilities might have been adequate.   Then I stumbled on Declude+SmarterMail and found something that could do what I thought every spam filter should do - multi-factor exceptions.   Then its extensibility opened up other possibilities.

WannaCry made it clear that spam filtering is critical to the future of civilization, but our defense technology looks too much like the stone age.   I believe there may be a huge opportunity for the vendor that understands the problem and is able to articulate a solution to it effectively.  At least I hope so.