If you have a backup of your Bayesian filter dat files from 15.4.6151 I would try restoring those and restarting SM.

 

You can find these files in \Smartermail\Service as:

 

    bayes.dat

    bayes_default.dat

    bayes1.dat

    bayes2.dat

 

If that still doesn't resolve the issue you may have to disable Bayesian Filters until you get a response from SmarterTools themselves.

Have you noticed anything specific to gmail, wherein more number of mails to gmail users are getting delivered to spam folder, post last couple of SM 15.x updated i have been noticing that many customers are complaining about mails getting delivered to spam folder and more of them are gmail recipients, emails with just test word and clean smtp IP even getting delivered to spam.

Are we getting any update on this from SM Team on this, it seems to be some scoring issue with Bayesian filtering, emails with simple text is getting scores 10 by bayesian and this is post last two SM 15.x last 2 builds.

 

Thanks

Can we get some update on this thread from SM team ?

 

 

I have been facing bayesian issue off lately post last couple of 15.x builds and before that was ok, off lately lot of complaints of false positives.

Hey guys. I have an email in to SM support about this, as you guys are not the only people seeing this issue. I also had a customer of ours report it to me the other day. As soon as I receive a response, I will post an update here.

So, I did receive an update from SM Support, but they only said that the Bayesian filter needs retrained: 

I think you finally just reached a point where enough users selected "Mark as spam" to start skewing the results.  

Sometimes the Bayesian filtering does need to get retrained. Typically we see this when the end-users are marking messages as spam, but frequently not marking messages as not-spam. I would recommend resetting this by the following steps:
1. Stop the SmarterMail service and ensure mailservice.exe exits completely.
2. Navigate to C:\Program Files (x86)\SmarterTools\SmarterMail\Service
3. Delete the Ham and Spam folders, and also the bayes.dat file. If you see any Bayes1.dat or Bayes2.bat, etc please clear these as well.
4. Start the SmarterMail service
5. The bayesian filter will now start fresh. Based on your settings, the filter should kick in after 500 classifications of spam\not spam by your end users.

So, it appears that if there are not enough users marking messages as "Not-Spam" then your Bayesian filter can be skewed and send everything into spam. 

Hi Colton. Thanks for the info. I really find it hard to believe that this is what is really happening. You are the 3rd person that has reported this in the past week or so. I'm thinking that there may really be a problem with the Bayes. It just seems strange that it's happening to you guys all of a sudden. I'm hoping that SM support gets back with me soon. If I receive the same answer that you received, I will question it further because again, I really feel that there is more going on. Thanks.

As Ron mentioned above, we've seen this behavior for many years, nothing new here.  You can reset (retrain) it as listed above, but that only lasts for a few months.

 

Idea for Software Enhancement:  Users shouldn't have to mark legitimate messages as Not Spam. Instead, all messages delivered to the Inbox with a spam score of 0 or less should be automatically fed into Bayes to keep it trained.

I agree with your suggestion, Kevin. I'm still waiting on SM support to respond to my inquiry. I just want to make 100% sure that there is not an actual issue with Bayes.

Hey guys. SM support emailed me and said that they are working on this further with their dev team. I'm hoping to hear back with more info as soon as possible.

Hi Linda / Colton,

 

Any one of you have received any further update from SM on bayes.

i  have just sent them reminder on my open ticket.

 

Tks

Guys, here is the exact message I received from SM support about this...

 

I've had an opportunity to sit down with the SmarterMail team and demonstrate the behavior. Through our testing, we've identified a major flaw with the Bayesian filter implementation, as a result of our investigation, Bayesian filtering is actually going to be removed from SmarterMail, starting with SmarterMail17 and dropping support in all older versions as well. 
 
We can no longer offer support for the Bayesian filtering spam check since it is fundamentally flawed and cannot be corrected without a complete re-write. We strongly advise disabling the Bayesian filter check indefinitely. Please note that the performance of the Bayesian filtering within SmarterMail has been surpassed by other methods of spam detection such as Message Sniffer and Cyren. These two platforms leverage hashing functionality along with heuristically based analytics to more accurately detect and stop spam messages from being delivered.

Someone suggested using Message Sniffer. So since we used Cyren already, I turned on the trial and ran an analysis.

 

This scorecard only includes messages which were scored by both Cyren and Message Sniffer (aka I excluded the messages from today when I did not have Message Sniffer trial enabled.) This pie also has nothing to do with changing settings or scored values away from the defaults. 

 

The two little slices are the key parts for your consideration. 

Cyren flagged 7785 messages on its own.

Message Sniffer caught 6251 messages on its own.

 

Hope this helps!

The chart is nice. I'm curious to know if the messages caught / passed are all known spam or all messages combined. Would also be nice to know how many false positives were caught, etc.

For those of you that feel the need for Bayesian filtering and don't want the expense of Cyren or MessageSniffer (which do much of the same thing on their own systems) there is an alternative that is much lower cost.  You can use SpamAssassin In A Box by JAM Software to configure a fully operational version of SpamAssassin that works great with SmarterMail.  It does take some time to get the configuration right, but once that's done it can take over most or all of the spam filtering for you (and includes Bayesian filtering).  It only cost about $50 per year.

 

I used SpamAssassin In A Box for several years and had good luck with it.  I personally dropped it when I switched to MessageSniffer.  I had our inbound spam filtering as good with SpamAssassin In A Box as we get with MessageSniffer, but SpamAssassin is not the way to go for outbound filtering.  MessageSniffer is excellent for outbound filtering as well.

 

The ONE good tool that SmarterTools gives us for outbound protection (Abuse Protection - Bounces Indicate Spammer) hasn't worked for years.  If that one tool worked it would essentially solve a huge problem, but I have to face reality that they'll never fix it.

 

The point is that there is a choice.

 

I don't want to go off topic, but there was a comment that SM17 is going to use a different setup for DNS. I have no idea why SM is in the DNS caching or any third party service for any reason.  We ALL run SM on a Windows server and every Windows server has a very fast an accurate DNS server built in... free of charge. Most URIBL's and some RBL rate limit or block public or high volume DNS query servers.  I don't know of any reason any SM installation should use anything but the DNS server on the local machine.  There is no value of any kind for using a public or third party DNS resolver.

 

-Joe

Hi all,

 

As Linda mentioned above, we have removed the option for Bayesian filtering from SmarterMail 17.x (currently in BETA), 16.x and 15.x, due to its fundamentally flawed implementation. Please note that the performance of Bayesian filtering within SmarterMail has been surpassed by other methods of spam detection, such as MessageSniffer and Cyren Antispam. These two platforms leverage hashing functionality along with heuristically based analytics to more accurately detect and stop spam messages from being delivered. In short, Bayesian filtering is a relic of older days. You could consider it the horse and carriage compared to the automobile that is modern spam protection. 

 

That said, those who still wish to implement Bayesian filtering may benefit from reviewing eFa project, a third-party, open-source filter that includes Bayesian and dedicated Spamassassin.