Declude - Weights, Sniffer Integration, and RBL Questions
Question asked by Jay Dubb - 1/13/2022 at 3:30 PM
We just installed Declude, and after learning a hard lesson with Message Sniffer (way too aggressive with the default weight applied) we want to use abundant caution with Declude until we gain confidence in its accuracy.

In the Smartermail admin interface --> Antispam --> Spam Checks, under Declude settings, we can assign weights for Low, Medium, and High.  We set these to 1, 2, 3, respectively, so we can spot-check headers and see how Declude is scoring a variety of mail, but without it significantly affecting the overall spam score yet.

Problem:  Declude is applying significantly higher weights-- like 4 and 8.  How do we actually adjust the weights applied, if not in the Smartermail admin console?

(UPDATE:  Looks like that is set in the "DECLUDE TESTS" section, yes?  Lots to tweak in there.)

Next question is Message Sniffer integration.  We already have Sniffer installed and weighted.  Does Declude, with its callout to Sniffer, actually cause a message to be evaluated by Sniffer TWICE?  If so, how do we turn that off in Declude?

Finally, it appears Declude also checks RBLs.  We already have those set up as well.  Does that mean Declude will do an additional RBL lookup, on top of the one Smartermail is already set to perform?  

We want Declude in the mix, but don't want it duplicating the RBL and Sniffer checks that are separately being performed.

Advice is welcomed.

2 Replies

Reply to Thread
Douglas Foster Replied
Declude parses the whole config script on every message.   You just need to comment out the rules and actions that you don't want.

Declude has its own weight system.   Every rule has a match score and a non-match score.   Filter files have their own score which is used to determine if the rule is triggered or not.   When the filter file is match, this score is added to the rule score.   Then the score of all of the rules are accumulated.  You can trigger a Declude action based on either a rule or a weight.   Or you can do nothing and just pass the score back to SmarterMail.

In this description, I am using "match" and "non-match" for clarity.  Declude uses "Failed" for matched, because a filter rule that matches usually means that the message is failing inspection.   But of course, this is not always true.   In my config, some matches are unfavorable and some are favorable. 

One of the beautiful things about Declude is that it allows test mode.   You can configure lots of filters, but not create any actions.   Then you parse the log files to see which filters triggered on which messages.  This allows you to configure your exceptions before activation and action command.  With every other product, I have to turn on a rule and wait to see how much it breaks, which of course is bad for technical reputation and job security.

You can also use weights to make a rule score-neutral.   For simple rules, you simply set the match and non-match weights to zero.   In a filter file rule, I set the max and min weights in the file to a specific value (n), then set the rule weights to negate it (-n), to produce a net of zero.   Then the non-match weight is still zero.  The score remains unchanged whether the rule is matched or not.

Allow vs Block
Lots of supposedly smart products are dumb about allow rules.  When creating a rule to block an unwanted identifier, you just need to know the identifier -- nobody wants to impersonate a identifier with a negative reputation.   But when you want to create allow rules, you should match on the identifier(s) and a verification result.   The possible verification mechanisms are:   Source IP (assumed true), HELO and Reverse DNS names if they forward-confirm to the Source IP, SPF, and a rough approximation of DMARC.   Declude only provides some of these methods out of the box, but I was able to add the missing items using custom script filters.   Having a customizable product has been essential to my success.

Alternatives to Scoring:
Personally, I don't believe in scoring, despite the apparent widespread success of SpamAssassin.    My experience with scoring in commercial products is that it has mixed results.   I figure that the real goal of spam filtering is not to block one bad message, but rather to find and block the organization behind a bad message.   Ambiguous results get sent to quarantine.   Quarantine review determines whether I add an allow rule to correct the false positives or block rule(s) to prevent the bad organization from attacking me in the future.

What you need to build for yourself:
To review your messages, you need to hang onto them.  Neither SmarterMail nor Declude provide a message log and review interface for the system manager, so you need to figure out how to create one.   The simplest one is to copy the HDR and EML files to a folder, and then do some file management to keep the archive from growing out of control.   If you have a commercial product in addition to Declude, you may be able to do most log review from their.   You will also need log to do some log parsing.   Normal people use a tool like Splunk.   I did not have the Splunk option, so I parse into SQL.  It has the BULK option for reading a text file as if it was a database table, and it has higher throughput than I would have expected. 

I am a big fan of having an incoming gateway, running SmarterMail + Declude, (Free Edition) as a separate server.   The filtering issues for unauthenticated mail are a lot different than the filtering issues for authenticated mail.   Having a server dedicated to inbound filtering also makes log analysis a lot easier.   Since even the free version of SmarterMail requires Windows Server, some organizations may find that this "free" option is too expensive.

I am still using original Declude, which is frozen code.   If you find a bug, you will need to find a way to work around it.  I have found very few -- they have been minor and I have been able to work around them with customization.   Declude Reboot is the future.   Talk to the vendor to decide which version of Declude is right for you.

You could pay really big bucks for a cloud-based email filtering solution.   Lots of big organizations do so, yet have still become victims of ransomware.   So I am no fan of the big name vendors.  When I was shopping for a commercial solution, I was stunned by how many products could not do things that I thought minimal requirements, and the remaining options were outrageously expensive.   Declude does amazing things for those who are willing to put in the elbow grease needed to be successful.

Jay Dubb Replied
Thank you for that great reply.

We've only been running Declude for about 16 hours, but WOW, its accuracy is WAY better than Message Sniffer.  

There are several high-volume mailboxes across several domains from which we pull sampling, and hands-down Declude has been "almost" perfect in its assessments so far.  On many blatant spams-- ones which even a dead cat could detect as spam, that should have been tagged-- Sniffer let them pass with a '0' weight.  (Hello, Sniffer, are you even awake?)

Fortunately, Smartermail's built-in SpamAssassin added several points (but not enough alone to be spam tagged), but that combined with Declude's score kicked it up high enough to be tagged appropriately.  Sniffer missed it, but SA + Declude nailed it.

To be fair, there were a few messages that were definitely spam, and Declude and SpamAssassin added points to, but not enough to rise to the level of getting tagged.  But this time Sniffer did add its points, kicking it up high enough for tagging.

Initial Impressions:  
  • Message Sniffer alone is only mildly effective, and NOT worth the money if used alone.
  • Declude + SpamAssassin (built-in to SM) are reasonably effective, when used together.
  • For those on a tight budget, Declude + SpamAssassin are a good (not great) free solution.
  • Declude + SpamAssassin + Message Sniffer, as a trio, seem to be highly effective.  For those with a few available dollars in their budget, adding sniffer will boost tagging accuracy.
We run a Cisco Ironport for a deep-pocket client, and loosely speaking, Declude + SA + Sniffer are roughly on par with Ironport.  But Ironport addes other useful features like a quarantine, although it should given it's exorbitant cost.

So at this point, it looks like we will purchase Sniffer when our 30 day trial expires.  We think it adds "just enough" value to justify the cost.  And we'll switch to Mail's Best Friend as our new reseller going forward, as a way of saying Thank You to them for their efforts in keeping Declude alive and well for us Smartermail users.

Reply to Thread