I start from an assumption that all of us have legacy filtering system which are pretty good at filtering based on known reputations, but generally lousy at filtering content from senders with unknown reputation. The result of that gap is that unwanted messages are released to our users. SmarterMail-integrated Spamfoo steps into that gap.
Spamfoo enters the delivery process after all of the legacy spam filtering tools have decided that the message is probably acceptable and can be delivered. Spamfoo scores those messages and routes them to either the Inbox or the Junk Items folder. Regardless of the spam or not-spam decision, every message still gets delivered.
If the training folder is enabled, this opens up a wealth of feedback information. Some feedback arrives because Spamfoo sent it to the Junk Items folder, some arrives because the user moved a message to the Junk Items folder, and some arrives because the user moved a message out of the Junk Items folder.
Spamfoo uses feedback to create a personalization database for each user, and that feedback has some influence on eventual revisions to the global model, but the results are not shared across the organization to other users, even if the message is dangerous. The email security team needs to invest time and effort to review this feedback, even if resource constraints mean that only statistical sampling is possible.
When this review process detects a dangerous message, the appropriate response is to act to protect the entire organization. The optimal response has two components:
- A block on the identity of the responsible sender, to protect against future messages from that source. Determining the responsible senders generally requires expert analysis. Legacy tools should be able to handle sender blocks.
- A block on the attack pattern to protect against similar attacks using a different identifier. Spamfoo is well-suited to this purpose, so we need to create a Spamfoo personalization database for the organization.
It turns out that organization-level personalization is feasible. Spamfoo has two main API calls:
- The feedback API call creates or updates the personalization data for a specific user-id.
- The classification API call scores a message using the global model plus the personalization data for a specific user-id. If no personalization exists, it scores based on the global model alone.
The user-id can be any string that looks like an email address, it is not validated against actual accounts in SmarterMail. So organizational level filtering is implemented as follows:
- Assign a non-existent email address to represent the organization, such as system@mycompany.local
- When feedback review identifies a message with organization-wide importance, send that message to Spamfoo using the organization’s user-id.
- When population of the organization-level personalization seems adequate, create a new filtering step that evaluates messages using the organization’s personalization data. The results of the filter can be used to block messages that should not reach any users. To protect against false positives, messages intercepted using organization-level Spamfoo should be routed to a quarantine folder, where they can be reviewed and released if necessary.
Organization-level Spamfoo means that you invoke Spamfoo twice. The organization-level test is invoked by custom filtering logic, and it can be made conditional so that the test is bypassed for from low-risk senders with verified identity. The user-level test is still invoked automatically by SmarterMail when the Spamfoo spam check is enabled and the domain-level classification option is enabled.
The code that I posted previously has some errors, so I tried to delete that post, without success. SmarterMail Support was helpful for finding my bugs and clarifying ambiguity in Spamfoo's posted API documentation. I am currently using Spamfoo only on my inbound gateway, to do organization-level scoring without enforcement. Declude calls a Python script, and the Python script decides whether the Spamfoo check is neccesary or not. I have some challenges with collect enough data to compare legacy filtering results with Spamfoo results, but I am making progress. Once the Spamfoo recommendations are better than the legacy results, I will begin enforcing its results.