Hidden Markov Model
Question asked by Jay Altemoos - October 19, 2016 at 11:32 AM
Unanswered
Just curious if there are any plans to add the Hidden Markov Model filter to SmarterMail's spam filtering in the future? Not sure if it's already built in or not. All I saw under the spam filtering section pertained to Bayesian filtering.

3 Replies

Reply to Thread
0
User Replied
Can you explain a bit more about this Jay? What are you hoping to see implemented in SmarterMail? The bayesian filtering method in place currently emulates a hidden markov model implementation, as only the end state is typically available to the product. Each message is scored against a series of tokens, and a probability of spam is then generated and applied to the message. This means that only the end result is visible (spam, or not spam) and none of the tokens are directly available to the system or end user. What information are you hoping to see leveraged in the antispam process?
0
Jay Altemoos Replied
Hi Kyle. The reason I asked about the HMM is because the last anti-spam product we used before handing over anti-spam duties to SM had Bayesian & HMM working together. From my understanding on both was that Bayesian is good and traps spam effectively to a point. Spammers are getting crafty at deliberately misspelling words in emails and other types of spelling errors. Bayesian itself could miss these types of crafted emails thinking it's legitimate email when it's not. The HMM fills in that the gap that Bayesian may miss. I know that regex could also trap these types of situations, but something like HMM could auto learn along with the Bayesian filtering and work hand in hand. Making SM's spam filtering even that much more efficient.
0
User Replied
Hi Jay, thanks for filling me in on these details. I know that our Bayesian filtering functionality does in fact learn over time, but I'm not sure whether or not it can handle the scenarios you've outlined. I'll be doing some additional research on this, and will provide your comments and my findings to our development team. Thanks for the feedback Jay!

Reply to Thread