My first email filtering appliance was purchased on a consultants recommendation when I had no idea what I needed. It had some useful features like vendor reputation block list and blocking of executable attachments. It also had an option to block on SPF Failure. As I learned more, and understood that SPF could block malicious impersonation, I decided that it should be enabled. This exposed the first product defect:
- The only way to know the impact of a new filtering rule is to turn it on and watch what happens. When the dust settles, hopefully the system admin will still have a job.
So I turned on SPF Failure blocking, and had a problem within 24 hours. An active vendor, call him Example.com, was getting blocked because he had switched his hosting service to
AppRiver.com but had never updated his SPF policy. This exposed the second product defect:
- SPF checks are treated as just another content filtering rule, and there are only two ways to fix a filtering error: turn off the filter or whitelist the sender
So I considered the implications of whitelisting. All that I needed was a local policy to treat
AppRiver.com as part of the published SPF policy, so that messages from malicious sources would be blocked. But that was not an option. Instead, I had to:
- Allow Example.Com to send messages with executable attachments.
- Ignore malicious impersonation of Example.com, so that impersonators could also send messages with executable attachments.
Therefore, to defend myself against some malicious impersonation, I had to create a security hole to facilitate malicious impersonation of Example.com. That requirement was a bridge too far, so I turned off SPF checking and started to look for a vendor who could implement SPF overrides without creating a security hole in the process. I looked at a lot of products. Eventually, I realized that the problem went way beyond SPF filtering. Every whitelist was a security hole. Recently, a vendor gave me a name for the problem, the "Whitelist Paradox." They understand the problem so well that it has a nickname, but they cannot be bothered to spend your subscription money to fix it.
How easy is this to exploit?
- An attacker can assume that your whitelisted domains are vulnerable to impersonation attacks, especially if they do not have DMARC p=reject.
- An attacker can study your website to know your industry and your geography, then use that information to make some reasonable guesses about your most important correspondent organizations.
- The attacker can impersonate a series of those likely correspondents until his payload gets delivered, and in his hopes, deployed.
Who is vulnerable to this attack? Everyone whose filtering structure is limited to single-attribute rules, which is probably everyone who reads this post.
I have been so upset about this that I tried to file a CVE. They dismissed my submission, so the problem remains unacknowledged.
A Solution
These principles came into focus:
- To be safe, every whitelisting must be authenticated
- Any message source may require whitelisting
- Therefore I needed an architecture where any message can be authenticated by local policy.
But if every allowed message can be authenticated by algorithm or local policy, it is a short leap to realize that every accepted message should be authenticated by algorithm or local policy, whether whitelisted or not. The only obstacle is collecting data and configuring the local policy exceptions (which was generally easier than expected.)
My product search led me to Declude from MailsBestFriend.com.
- By design, it could collect a test result without taking action on that result, so I could test new ideas without breaking the inbound mail flow.
- It had a rules engine to begin implementing a complex local policy.
- It was customizable, so I could implement my own proprietary filtering policy.
Amazingly, the features that I could not buy at any price were achievable using a free product and some development effort.
I toyed with several designs for constructing the exceptions in local policy, and settled on this one:
- If the Mail-From address matches <string1> and the server name matches <strng2> and the server name is validated by forward-confirmed DNS, then the message is equivalent to SPF Pass. (On rare occasions, I match to IP address directly, rather than to verified host name.)
Note that authentication by SPF Policy says only one thing:
- The message is authentic, and the Mail-From address is free of malicious impersonation.
But authentication by local policy says two things:
- The message is authentic, and
- Messages from this sender are normally acceptable, which is why the local policy rule was created.
Eventually, I extended the design to support From-address authentication on all messages. It was a simple extension of the SPF solution:
- If the message's From address matches <string3> and the Mail-From address matches <string4>, and the Mail-From domain is authenticated by SPF-Pass or equivalent, then the message has the equivalent of DMARC Pass.
Within the last few weeks, I learned that the Reply-To reputation needs to be evaluated when it is different from both the Mail-From address and the message From address. This was also a logical extension of what came before.
- If the Reply-To address is <match-string5> and the message From address is <match-string6>, and the message From address is authenticated by DMARC Pass or equivalent, then the Reply-To address is acceptable.
Final note: Blacklisted identifiers do not need to be verified at run-time, so blacklists still work with a single-attribute filtering rule.