Message sniffer not too bad, Cyren awful

Currently using "Powerful Spam Filter for Windows - SpamAssassin in a Box | JAM Software" https://www.jam-software.com/spamassassin_in_a_box

Viv

Viv,

How does the SA in a box works out for you.

Does it work well?  Is this something you would recommend?

Cyren is basically the same as hiring a a really dumb monkey to manually scan your messages with a stamp... Sniffer is hiring a semi-intelligent monkey with a stamp...

SA in a box works well for us now, there was a bit of trial and error in the beginning but after a few days it settled down and is very efficient.

There is a trial version available.

Viv

We use Message Sniffer with some additional trusted RBLs/URIBLs and it works great for us.

Message Sniffer as a standalone has worked well for us for many years.  We used to vector out to it using something called MxGuard which is no longer available so now use Declude.  Our experience with Sniffer in terms of reliability has been stellar.  It runs as a service and is never the source of mail problems.  It's Good Bad and Ugly RBL  (GBUdb) is excellent and well maintained.  We paid a very reasonable price to have Mails Best Friend do the initial Declude installation.  I think you would end up at the same place downloading it for free and installing it yourself but it seemed only fair to at least get them something for continuing to make Declude available and, although we don't have a lot of contact with them, we do try to throw as much business their way as we can.  Contact is first rate when you do have a question and they are great contributors to the community.  As for Declude it's primary use for us beyond the original configuration is to handle the vector out to Sniffer but it does have some some live configuration capability that can be quite useful as a quick fix if something is managing to get through in a big way where a simple IPSEC block won't do the trick. It's done though text files, not a GUI, but it doesn't take long to get a handle on which ones contain the functionality you need.  Although we don't do much of it, you can also create special handling or exceptions right down to the domain (possibly even user) level.

I can't compare it to SA in a Box mentioned elsewhere here although the mention did get me to go take a look at it again.  My gut feel is that we'll probably implement SA in a Box  as a trail but not look to it as a replacement for Sniffer,  just an extra layer if it does improve things.

If you want to get a significant reduction in spam quickly for your new client I wouldn't hesitate to suggest that you take advantage of the trial period on Message Sniffer and install it alongside Declude.  I think you will find that it will at least buy you a little breathing room to look at other tools and that you may end up finding that it really does all you need.  I expect there will be enough of an improvement immediately, without the need for a lot of tuning, that your client will see that you are taking it seriously and working on it.

Thanks to everyone for their helpful replies so far.  

We are doing the 30 day trial of Message Sniffer now, and when reviewing the logs, it appears to be catching a lot.  Since it's integrated with SmarterMail, it was a simple click of the ON switch, and a little tweaking on the weights for tagging and blocking.  The price is low, so we will convert to a paid license after the free period.

We're still trying to make sense of how much more (if any) Declude would bring to the table.  If Declude would noticeably increase filtering effectiveness, it may be worth it.  But if we're looking at less than 1% improvement over Sniffer alone, well, maybe not.  The server handles a little over 2.1 million emails per month, so we have to consider loads each layer of filtering adds, especially with the significantly higher loads MAPI and EAS have added.  (This server now requires as much RAM and CPU as our Exchange Server Enterprise edition, which came as a bit of a shock to us.)

Comments are welcomed on how much more Declude and/or SA in an Box would add over and above Sniffer alone.  Our minds are still open to feedback and your own experience/observations.

Jay, my suggestion is Message Sniffer for now.  We believe more can be done around SPAM and have a couple things in progress and plan to release more information in the near future. 

In regards to SmarterMail and Exchange resource usage.  SmarterMail uses 10% or less compared to Exchange and why hosting companies, ISP's and Service Providers use SmarterMail... it is all about the volume we can handle and how little we cost.  

If you are seeing high usage, you will want to make sure you are on the the latest build.  EAS doesn't have much of an impact on SmarterMail and we have servers with 1,000's of concurrent EAS connections.  But with MAPI, we did discover some clients that are in bad states for various reasons could denial service attack SmarterMail and would cause higher resources.  We knew Outlook 2013 was a disaster and why we chose not to support it but we weren't aware there were some other scenarios that could cause it with current Outlooks.

Lot more information in this thread and any questions related to performance and that release, lets discuss it there: https://portal.smartertools.com/community/a94536/build-8025-feedback.aspx

I use Declude without Message Sniffer, because I already had other products in place to do content filtering.   Before implementing it, Linda and I compared notes about one particular attack, and her Declude+MessageSniffer combination blocked all attacks while my spam fitlers let them through.   So MessageSniffer may be better than average at spam filtering.

What I love about Declude is the ability to customize it.   I have done these customizations:

- replaced the embedded SPF test with an external one that works better, 

- added code to do a poor man's implementation of DMARC, and 

- implemented SQL integration.   

SQL integration:

- Metadata from every message is pushed into SQL in real time.

- 80% of my filtering logic is implemented with SQL queries instead of text filters.  SQL queries use indexing for good performance even with thousands of filter rules, and unique keys prevent duplicated entries. 

- Additional data is pulled into SQL after-the-fact by parsing the Declude log files.

- I use ad-hoc SQL queries to determine how filter rules should be improved.

Unwanted messages come from unwanted sources, so my general approach is to turn any detected spam into a permanent block on the source.  For direct messages, this is usually a block on the host DNS name and the source IP address.  For service provider messages, it becomes a block on the From domain.   Finding a spam tool that can filter on both HELO and ReverseDNS should be easy, but has actually difficult.  Finding products that can filter on the From address has also been difficult.   Declude has poor embedded support for From address filtering, but customization allowed that weakness to be overcome.

For a particularly problematic email service provider, I have a three-way filter.   Known-good From domains are allowed, known-bad From domains are blocked, and unrecognized From domains are quarantined.  Declude is the only product, at any price, that could do this for me.  Two of my commercial products do not examine the From address as all.

 

My inbound flow goes to a incoming gateway based on SM+Declude, then to two different commercial spam filters, then to my SmarterMail main server.   Declude does the heavy lifting, but the commercial spam filters do the fancy content filtering that only they can do.   Regular tuning is needed, but the overall result has been very satisfactory.

Also, you and your clients should remember that most email-based attacks use web links for deploying the payload.   Email filtering is important, but a good web filter is probably more important.  

I have a day job, but am willing to share my code, and offer assistance on a moonlighting basis.

UPDATE:  With the addition of Message Sniffer, we've been flooded with complaints about false positives.  We continue to reduce the weight on Sniffer, trying to find a balance, but are concerned that we will have to turn the weight down so far, it may not be particularly useful.  Some examples of legit messages being tagged as spam:

We feel that if a recipient marks a sender as Trusted, then any spam tagging should be removed before final delivery to that recipient's mailbox for future emails.  This is what our customers are telling us.

Jay,

That was our biggest issue with Sniffer as well. Constant false-positives, same exact message types as well. It's just not worth the price tag, same as Cyren. Would love to see SmarterTools partner with other providers. We did ask our VadeSecure rep to reach out to SmarterTools a bit back.

The FP issue was enough of a reason for us to move to SA in a box. I'm surprised how stable it has been to date, now I've spoken it will invariably now play up.

Viv

We decided to install Declude as a trial, to see if it's any better than Message Sniffer.  Results have been excellent, and our comments are in this thread:

The quick version is, Sniffer is just too problematic by itself-- too many false problems, AND too many blatant spams slipping through un-tagged.  Two extremes; too aggressive or not aggressive enough.  There doesn't seem to be a middle-ground.  

In just the few days we've been using it, Declude has impressed us.  It was very easy to install and configure.  Declude + SpamAssissin (Smartermail's built-in version) catch most of the spam.  We've turned the Message Sniffer weight way down so that it's not enough to generate false positives on its own, but IS enough in certain cases to push a message over the tagging threshold if Declude + SA alone don't quite add enough points.

Jay:

I'm glad this is working out for you.  All of my original comments were based upon using it in conjunction with Sniffer.  I've looked at the direct integration several times.  I've always found the people at Mails Best Friend to be knowledgeable and candid.  They suggested staying with it and using it in conjunction with Declude and had some specific reasons for that.  I didn't doubt them but your experience with this has me even more convinced.

I did look at replying to some of the problems you were having and spent some time looking at what it was that Declude was doing that made our mileage with Sniffer so much different than yours.    I decided that simply suggesting that the weighting system was where the magic was happening was likely a little superficial and I don't claim to know Declude well enough to have been able to get into it and then defend my observations at a deeper level.  When I do wade into Sniffer to try to do some simple quick fixes I've almost always found some capability in there that will do what I need and with a bit of searching through old documentation I've had some confidence in the solution.

I twill be interesting to see what sort of functionality the refreshed version of Declude will bring to the table.  I've seen mention of it being tested currently in a few places here and elsewhere.

We're not thrilled about ponying up the $$ for Sniffer when our trial version expires next week, considering it's nowhere near good enough to stand on its own merits.  (What does it say when free products like Declude and SA perform substantially better than a paid product?)  But, we onboarded a really big client in 4Q'21 (well over 1000 users all with MAPI and EAS) and want to do everything within reason to keep their experience good.  If that means forking over five or six hundred bucks a year for a little extra spam filtering (emphasis on "little") above the free tools, then so be it.  Cost of doing business.

But I stand by what I said in the other thread about Declude + SA + Sniffer being "almost" as effective as IronPort for filtering-- without IronPort's heavy pricetag.  (we run IronPort for one of our clients and it is obscenely expensive)

Jay:

Please don't misunderstand this as an attack on what you are saying or even disagreement, perhpas just a different perspective.  Any rational I offer will be emperical.  I haven't analysed the end to end process in the recent past to offer real statistics against the threats which are ever evolving and changing.  One issue we are clearly in step on is not loosing these high user count clients after betting our reputation and moving them onto SM.  SM is also evolving and, for all but the most spendthrift, I don't think you can make a business case for accepting the differences in the cost between it and Exchange based solely on the merrits of SM's lower cost.  The case can be made on the overall solution and a component part of that, and high perceived value (or problem if you turn the coin over) is anything arriving in the inbox that is unwanted.  IronPort would definitely negatively impact the price for the service.  With the spam problem resolved I really de believe that SM will evolve to support mail clients in a way that truly does make it an Exchange alternative.

Cobbling together something "almost" as good, to use your metric, is definitely possible and I've tried several paths with reasonable results over the past couple of decades.  When I've hit the target there has always been more than one component at play.  One of the things I've needed to consider is the amount of input and maintenance required to keep things tuned to a reasonable level to achieve the result.

Setting aside detecting a virus for a moment, I think there are three high value areas - currently known offenders, technically flawed header information, and content.  Content has always been a slippery slope, especially at low cost, if used beyond being a signature to identify something.  I listed those in the order that I've come to believe are most effective for us:  some sort of real-time connection refusal based upon a known current attack, technical problems that suggest the sender may be undesirable and, to a much lesser extent, content.

I wish I had the time to really dive into what Declude brings to the table at a deeper level rather than just jumping in searching for something that might be helpful or digging to find a few of the legacy things that may be problematic when we need to.  I'm sure the end result would be even better but I don't have that kind of time and it sounds like Mails Best Friend may still be working on some sort of refresh.

So, here's my slightly different perspective.  I am confident that Declude would not achieve the results on its own that it does with Sniffer being part of the mix.  I wasn't expecting it to be so dramatic but followed this thread in hopes that you would post current real world results against a large userbase after implementing the embedded solution. I was hopeful but wasn't expecting that you would subscribe following the trial.  I don't think Sniffer is the solution in and of itself. I believe it is an essential part of the overall solution we have arrived at.  Some of the magic Declude brings to the table is incorporating the Sniffer results within the rating rather than trusting them implicitly as the rating.  As to the overall cost, if you consider the three components you list - Declude + SA + Sniffer - are they proportionate to where the successes are achieved within the solution?  I am sure they are not, but it is a solution and the relative costs of the component parts to achieve it are not something I look at much any more.  There is a cost to having me dragged down that rabbit hole, looking at some of the other fine work being done.

IF MBF are working on a refresh of Declude and IF there is enough interest from the community to support it by contributing to the cost by paying to use Declude, perhaps an even better solution could be incorporated into a bundle that became an integrated product and we'd see only the one price tag much like we do with other services.  I haven't looked at the bundling costs for Sniffer in many years but suspect that MBF could offer a more effective hybrid solution than the simple SM integration option for Sniffer and that there would be profit in it.  That said, Linda is quite a contributor to the community and it wouldn't surprise me to find that there is simply  a posting showing how to do it and that, if you want to use a refreshed version of Declude to avail yourself of the advice, there will be a charge to use the refreshed Declude alongside the other components, including Sniffer.

For the time being it seems you have found a solution that works at a price you can rationalize.  If you continue your search I'd definitely be interested in anything you come up with that does a better job at a similar price or an equivalent job at a lower price and that doesn't require a lot of input and review in constant tuning.

Thanks for all you've posted so far.

Some thoughts on obtaining success with email filtering, with references to Declude as appropriate.

Everything is about identifying and blocking malicious sources.

If you are having good results with Declude alone, it must be because of the many RBLs that it can use.   Declude has no proprietary knowledge of its own.

A malicious source can be a server organization, the mail system operator (SMTP domain), or the end user (From domain or full From address)    

If the problem is the server, you want to block all servers from that organization.   You don't know all of the IP addresses used by the attacker, but you can use a block rule based on "host name ends with <value>" as well as "SourceIP=<value>".  Spammers don't bother to change host names as often as I was told to expect.   Declude has built-in capabilities to filter on Reverse DNS name, but it does nothing with HELO.    I have used the HELO string from the HDR file, in custom scripts, to offset this deficiency. 

If the attack comes from a malicious mail system owner using a legitimate server platform, then you block on MAILFROM, the SMTP address.  Declude does this easily.

A significant number of attacks come from legitimate email service providers who have unethical client organizations.  For these, you need to be able to block on the From address.    Declude has no specific support for filtering on the From address, but again I extract the From address from the HDR file using a custom script.  Parsing an email address out of the message file can be very complicated.  SmarterMail does a pretty good job, but if it gets confused, it punts by repeating the SMTP MailFrom address in the HDR file line that is supposed to be the From address.   I have written some custom code to parse the file myself, to correct specific scenarios where SmarterMail gets confused, but there are deficiencies in my code as well.   I have reported the known parsing failures to SmarterTools, and they are in the queue for future bug fixes..  

One particular email service provider has a particularly difficult client mix.    Some of his clients are sending critical traffic like password resets, and others are sending fake bank account messages.   They seem to be doing better at client control in the last 6 months, but I still have them on a short leash:   Known-good client domains are allowed, known-bad client domains are blocked, and uncategorized client domains are quarantined.   Not many products, at any price, can give you this much control.

For content filtering, I rely on a commercial product.   For sender authentication, I use custom code to implement a better version of SPF and a crude approximation of DMARC.   In both cases, the real goal is to use these checks to identify malicious senders, then add another rule to block that identifier.

Consider that if you identify a malicious organization, you ideally want to block all references to that domain name, including:   Server HELO name,  Server Reverse DNS name, SMTP address, and From address.   If Declude could parse URLS in the message body, I would also have it parse the URLS for blocked domain names.   Someone could probably build a custom script to parse the message body for URLs, but it is above my talent pool so I leave URL filtering to the commercial spam filter.

As I was implementing Declude and building my rule set, the text files quickly grew to 1000s of entries, and the number of text files was steadily growing as well.   That's why I moved metadata into SQL so that some of my filtering could be done with indexed tables instead of text files, especially since every text file is re-parsed for every message.

The beauty of Declude is its customization.