I have SpamFoo working on my incoming gateway to perform analysis of high-risk messages, which has required doing a pretty deep dive into the product.
Concepts:
Spamfoo provides two types of scores:
- Risk: Is it spam or not? This involves a binary result and some additional elements that indicate confidence of the result.
- Classification (Category): If it is not spam, is it Primary (e.g. personal), Transactional (automated confirmations), Promotions (unsolicited), or Updates. I am not certain what is intended by "updates", and have not tried to figure it out.
Mechanism:
Spamfoo has three functions: One function is use to classify / score a message, one function submits risk feedback (optionally with category corrections), and one function provides category corrections only.
All functions are associated with a userid. A userid with no feedback will be scored using the default model. As soon as feedback is provided, a personalization database is created to store that feedback. Thereafter, scoring of messages for that userid will use a combination of the base model plus the personalization.
SmarterMail processing:
SmarterMail always invokes Spamfoo using the context of the message recipient.
When Spamfoo is enabled, Smartermail uses the spam / not-spam result to determine whether the message goes in the Inbox or the Junk Email folders. When the user moves messages between Inbox and Junk, SmarterMail sends risk feedback to SmarterMail to create a personalization database for that user.
When classification is enabled for a domain and a user, incoming messages are associated with a Classification / Category which is independent of the folder structure. Old email is also processed to provide a classification, which has created performance problems for some environments.
I do not know if SmarterMail provides a way for users to change a message's classification, as I have not been interested in using Spamfoo at the user level. I do not think most users would spend time correcting classification even if the feature were present. Time will tell whether users consider the classification / category values to be useful or not.
Spamfoo does not assign a category classification when the message is considered to be spam, so a message that is moved from Junk to Inbox will continue to have no category classification. I think it would be nice to know if a spam message imitates a personal message or imitates a transaction confirmation. This would be an interesting topic for future discussion.
The Spamfoo dashboard is only visible within SmarterMail when classification is enabled on a doman, but it can be opened outside of SmarterMail, using a localhost connection, with some difficulty.
Spamfoo settings in SmarterMail:
- Spamfoo service must be started (System... Manage... Troubleshooting... Services). This applies to both default configurations and incoming gateway configurations that use custom logic.
- To cause Spamfoo to choose between Inbox and Junk Email as the delivery folder, enable the Spam Check option for Spamfoo
- To see the Spamfoo dashboard, enable Spamfoo classification on the domain. Also choose a setting to control how aggressively SmarterMail applies classification / category values on old messages. When enabled, this setting will creates the classification / category view structure in each user's inbox.
- To disable the classification / category view structure for a specific user, change the setting on the user's property page.
Extending Spamfoo to other use cases
The Spamfoo architecture is extremely flexible.
To configure system-level filtering personalization, you need to submit feedback using a userid that represents the system administrator, and then invoke Spamfoo using that system adminstrator userid. (The userid needs to look like an email address, but it does not need to exist because no validation is performed.)
To configure domain-level filtering personalization, you need to submit feedback using a userid that represents the domain administrator, then invoke Spamfoo using that domain administrator userid.
This strategy can be configured to any level of complexity. You could have a userid that represents the HR department, and create a personalization structure just for them.
Considerations for using extra filtering layers:
- These extra layers of filtering can be easily invoked from Declude or equivalent. On my incoming gateway, I invoke Spamfoo using a userid that represents the system administrator, and only do so for a subset of messages. I do not use Spamfoo on my main server because I do not want the overhead of scoring every message.
- Spamfoo returns scoring results, but you need to decide what to do with the results. I am currently using the results for data collection purposes only. When Spamfoo has enough feedback to provide high-confidence results, I expect to a high risk score to send a message to quarantine.
- To use system-level or domain-level filtering, feedback data collection is the most difficult part of the problem. You need to figure out who is qualified and authorized to provide accurate feedback, how they will be provided the data needed to make those determinations, and how they will access the EML files needed to submit the feedback. This is the most difficult part of configuring Spamfoo as an incoming gateway tool.
- Each layer of filtering requires a separate call to Spamfoo for classification in the context of each userid. If you use more than one filtering layer, you will need to decide how to prioritize those results to select a final action on the message.
- To minimize processing costs, it is reasonable to be selective about which messages are submitted to Spamfoo at each userid. Most of my messages are flagged for Whiltelist, Blacklist, or Quarantine by my legacy filtering logic, so I only invoke Spamfoo on the subset of messages which have unknown reputation.
- The code for using Spamfoo within Declude is pretty simple. Support helped me resolve the issues that I could not solve using the Spamfoo documentation alone.