SmarterMail provides both Mail Server and Spam Filter features. I am accustomed to thinking of these as independent solutions. I currently have SmarterMail hidden behind two different spam filters from two different vendors. I am increasingly frustrated by their inadequacies, so they shall remain nameless. I know that I could configure a second SmarterMail system in Smart host mode to be a dedicated gateway, but I don't see it as an adequate solution to the problem either. Below are the requirements that I plan to use for my next spam filter procurement. Unfortunately, I am not sure the product exists. Since SmarterTools wants to be in this space, I am sharing it here first.
Requirements for Fighting SPAM
Message Log with Query Interface
A Message Log Viewer helps to identify possible SPAM. When investigating complaints or penetrations, it becomes important to determine what other message were received from the suspicious source. Without a good view into the message log, patterns become impossible to identify.
To support real-time analysis, the spam filter should retain at least 30 days of messages for analysis using its query interface. Then it should allow export of the logs to a relational database for longer-term retention and more sophisticated analysis.
Blacklisting
A typically spam investigation will typically start from a suspicious message, and then look for related messages by searching the message log for:
- All message with the same IP address or within an adjacent IP address range
- All messages with similar Reverse DNS name suffixes
- All message with the same FROM address (most spam filters capture the SMTP from, rather than the From Header. The SMTP From is probably more useful for organization-level threat analysis. Ideally, both From entries should be available for searching.)
The message log viewer should make this type of navigation as simple as possible, so the complete range of blocks can be determined and the complete set of mishandled emails can be identified.
Once the three-part analysis is complete, the corresponding blacklist entries need to be made. Because spammer organizations do not use stable configurations, most actions should involve four actions:
- Block the offending IP address or range
- Block the offending Reverse DNS suffix
- Block the offending from addresses
- Notify the vendor to update his database
The three-part block is a logical “OR”, any one match is sufficient to block the message. This helps to ensure that future messages are blocked even if the sender changes identity or servers.
The ideal spam filter should integrate the search, reporting, and blacklisting into one seamless process.
Whitelisting
I have worked with spam filters that consider whitelisting to be the reverse of blacklisting – the user can pick one criterion, and then it will allow all traffic for any messages that match the rule. They make one exception, messages with confirmed malware attachments are blocked even if otherwise whitelsted.
Anyone familiar with the email infrastructure should recognize this as folly. Assume that I want to whiltelist messages from “ExampleCompany.com”, which is hosted in the cloud using services provided by “HostingExample.com” . If I whitelist all messages from “@ExampleCompany.Com”, I am vulnerable to anything that has a fraudulent sender address (which is common). If I whitelist all servers at “HostingExample.com”, I am vulnerable to all messages sent from other clients of that hosting service. The whitelist must be multi-part: From a particular email domain and from an expected set of servers.
Additionally, I have been frustrated with spam filters that have an all-encompassing whitelist action. If I am whitelisting something because the sender has an incomplete SPF entry, I really only want to create an exception for SPF, not necessarily for every possible rule.
If I am going to whitelist a message, I also need to know all of the reasons that the message was blocked. The typical spam filter will process rules until a block condition is detected, then stop. If I have followed the blacklisting strategy listed above, removing the blacklist may require multiple block rules to be modified. Therefore, the spam filter should provide a simulation tool so that I verify that a reference message will be allowed the next time that it is attempted.
Policy Test Mode
SPF is an example of a defense where the ramifications of enforcement are difficult to predict. A good spam filter should allow me to collect SPF results on messages without acting on those results. After a period of data collection, the system administrator can determine what exceptions need to be configured before the policy is activated. Test-mode data collection should be available for at least these defense mechanisms:
- SPF
- DKIM
- DMARC
- New RBL sources
- Forward-Confirmed Reverse DNS
Previously-Dispositioned Messages
A blacklist investigation typically involves these steps:
- Review the message log, particularly “Allowed” messages, for suspicious entries
- Identify a problem and take corrective action to blacklist or whitelist those messages in the future.
- Continue reviewing the message log for other issues.
To support the “continue” step, it would be highly desirable to filter the message log for “Allowed” messages that have not had a disposition change as a result of the revised blacklist/whitelist rule set.
Here is one way to achieve this result:
- Using the integrated search-and-blacklist process described earlier, the user proposes a set of new rules, and then evaluates the message log to review all of the historical messages that would be affected by the new rule(s).
- The user is pleased with the results of this analysis, and approves the changes. The spam filter activates the new rules, and simultaneously flags the historical messages with the disposition that would have been applied under the new rules.
- The user starts a new investigation by selecting message that “would be allowed under current rules” rather than “messages that were allowed under previous rules”
DMARC
DMARC was invented because DKIM and SPF did not work as standalone methods. The technology involves enforcement rules, feedback data collection by the receiving organization, which is then transmitted to the sending organization for evaluation. The feedback evaluation process is unrelated ot the spam filtering effort, and is financially prohibitive for most smaller organizations. The feedback collection and transmission feature is desirable but not strictly required. However, DMARC is mature technology and there is no excuse for a spam filter which cannot evaluate messages based on DMARC instead of considering SPF and DKIM individually.
Authentication-Results header
RFC 7601 lays out an architecture for a trusted spam filter to pass authentication test results to a trusting mail server or mail user agent. While there is a shortage of mail user agents that support this feature, the concept is important for integrating the end user into the spam defense system.