Declude Score No Longer Being Added To Spam Score

Problem reported by Scarab - 3/8/2024 at 12:53 PM

Submitted

When running Declude & Spam Checks in SmarterMail on Incoming Gateways the Outgoing Score is no longer totaled correctly and as such the SMTP Blocking Outbound Weight Threshold is bypassed and the TotalSpamWeight Score is incorrect when passed to the SmarterMail primary.

Here is an example, where Declude calculated an Outgoing Score of 40, SmartHostSpamWeight showed a score of 50, and the SmarterMail RBL & URIBL scored a 10, yet the SmarterMailTotalSpamWeight score ended up being a 10, ignoring Declude Outgoing Score and SmartHostSpamWeight entirely.

X-Declude-Scan: Outgoing Score [40] at 19:15:26 on 29 Feb 2024
X-Declude-Tests: BADHEADERS [8], BASE64 [4], FILTER-NOSENDER [5],
FILTER-COUNTRY [10], FILTER-LOTTO [20]
X-SmarterMail-SmartHostSpam: X1NQRj0wLFBhc3N8X0RNQVJDPTAsbm9uZXxSZXZlcnNlIERucyBMb29rdXA9MCxQYXNzZWR8TnVsbCBTZW5kZXI9MCxwYXNzZWR8X2ludGVybmFsc3BhbWFzc2Fzc2luPTEuNDoyfF9ES0lNPTAsTm9uZXxfY3VzdG9tcnVsZXM9fFNQQU1SQVRTPTB8U09SQlM9MHxHQlVEQj0wfFNVUlJJRUw9MHxQUk9URUNURUQgU0tZIC0gUmVkPTB8SE9TVEtBUk1BIC0gQmxhY2tsaXN0LCBIT1NUS0FSTUEgLSBCcm93bmxpc3QsIEhPU1RLQVJNQSAtIFdoaXRlbGlzdD0wfFVDRVBST1RFQ1QgTGV2ZWwgMT0wfElBREI9MHxETlNXTD0wfEJBQ0tTQ0FUVEVSPTB8U0VOREVSU0NPUkU9MHxNQUlMU1BJS0UtWj0wfElYPTV8UFJPVEVDVEVEIFNLWSAtIFllbGxvdz0wfFVCTD0wfFdQQkw9MHxNQUlMU1BJS0U9MHxNU1JCTC1WaXJ1cz0wfFNFTS1CTD0wfEhPU1RLQVJNQSAtIFllbGxvd2xpc3Q9M3xVQ0VQUk9URUNUIExldmVsIDI9MHxTVVJCTCAtIFNwYW1Db3A9MHxTUEFNSEFVUyAtIFpFTj0wfEJBUlJBQ1VEQT0wfEJPTkRFRFNFTkRFUj0wfFNFTS1CUz0wfE1TUkJMPTB8TUFJTFNQSUtFLVdMPTB8U1VSQkwgLSBNdWx0aT0wfFNFTS1VUklCTD0wfFNFTS1GUkVTSDE1PTB8VVJJQkwgLSBCbGFjaywgVVJJQkwgLSBHcmV5LCBVUklCTCAtIFJlZD0wfFNQQU1IQVVTIC0gREJMPTA=
X-SmarterMail-SmartHostSpamWeight: 50
X-SmarterMail-SmartHostSpamSalt: -1455312627
X-SmarterMail-SmartHostSpamKey: 968927924
X-SmarterMail-Spam: Null Sender: 0, SPF [Pass]: 0, DMARC [none]: 0, Reverse DNS Lookup [Passed]: 0, ISpamAssassin [raw:1.4]: 2, DKIM [None]: 0, SPAMRATS: 0, SORBS: 0, GBUDB: 0, SURRIEL: 0, PROTECTED SKY - Red: 0, HOSTKARMA - Blacklist, HOSTKARMA - Brownlist, HOSTKARMA - Whitelist: 0, UCEPROTECT Level 1: 0, IADB: 0, DNSWL: 0, BACKSCATTER: 0, SENDERSCORE: 0, MAILSPIKE-Z: 0, IX: 5, PROTECTED SKY - Yellow: 0, UBL: 0, WPBL: 0, MAILSPIKE: 0, MSRBL-Virus: 0, SEM-BL: 0, HOSTKARMA - Yellowlist: 3, UCEPROTECT Level 2: 0, SURBL - SpamCop: 0, SPAMHAUS - ZEN: 0, BARRACUDA: 0, BONDEDSENDER: 0, SEM-BS: 0, MSRBL: 0, MAILSPIKE-WL: 0, SURBL - Multi: 0, SEM-URIBL: 0, SEM-FRESH15: 0, URIBL - Black, URIBL - Grey, URIBL - Red: 0, SPAMHAUS - DBL: 0
X-SmarterMail-TotalSpamWeight: 10
X-SmarterMail-SpamAction: Low | PrefixSubject

The message should have been quarantined by the Incoming Gateway for exceeding the Outbound Weight Threshold of 45

Note: This isn't new to Build 8832, as it has been doing this for a while now, quite possibly a year.

Here are my other settings on the Incoming Gateways:

If these settings are no longer correct in recent versions of SmarterMail then what should I be setting these settings to for Incoming Gateways that do the Declude, Spam Filtering and SMTP Blocking?

7 Replies

Reply to Thread

Roger Replied

3/8/2024 at 1:47 PM

I see this the first time, where do you find this settings?

Scarab Replied

3/8/2024 at 1:52 PM

@Roger,

Those Spam settings are from SETTINGS > GATEWAYS / FAILOVER when you select an enabled Gateway.

Scarab Replied

3/8/2024 at 2:08 PM

Here is another example:

X-Declude-Scan: Outgoing Score [22] at 12:53:25 on 08 Mar 2024
X-Declude-Tests: SUBCHARS-55 [1], SUBCHARS-60 [1], SUBCHARS-65 [1], NONENGLISH
[5], FROMNOMATCH [2], FILTER-BULK [4], FILTER-FINANCE [5], FILTER-SCHEME [6],
FILTER-STOCKS [12]
X-SmarterMail-SmartHostSpam: X1NQRj0wLFBhc3N8X0RNQVJDPTAscGFzc2VkfFJldmVyc2UgRG5zIExvb2t1cD0xMCxGb3J3YXJkTWlzbWF0Y2h8TnVsbCBTZW5kZXI9MCxwYXNzZWR8X2ludGVybmFsc3BhbWFzc2Fzc2luPTI6M3xfREtJTT0wLFBhc3N8X2N1c3RvbXJ1bGVzPUJBRFNVQkpFQ1Q6IDc7fEJPTkRFRFNFTkRFUj0wfE1TUkJMPTB8U1VSQkwgLSBTcGFtQ29wPTB8TUFJTFNQSUtFLVdMPS01fFNFTkRFUlNDT1JFPTB8U1BBTUhBVVMgLSBaRU49MHxTT1JCUz0wfFNFTS1CTD0wfFBST1RFQ1RFRCBTS1kgLSBZZWxsb3c9MHxVQkw9MHxNQUlMU1BJS0UtWj0wfEJBUlJBQ1VEQT0wfFVDRVBST1RFQ1QgTGV2ZWwgMT0wfFNVUlJJRUw9MHxTUEFNUkFUUz0wfE1TUkJMLVZpcnVzPTB8SE9TVEtBUk1BIC0gWWVsbG93bGlzdD0wfElBREI9MHxIT1NUS0FSTUEgLSBCbGFja2xpc3QsIEhPU1RLQVJNQSAtIEJyb3dubGlzdCwgSE9TVEtBUk1BIC0gV2hpdGVsaXN0PTB8V1BCTD0wfFNFTS1CUz0wfFBST1RFQ1RFRCBTS1kgLSBSZWQ9MHxNQUlMU1BJS0U9MHxJWD0wfEROU1dMPTB8R0JVREI9MHxCQUNLU0NBVFRFUj0wfFVDRVBST1RFQ1QgTGV2ZWwgMj0wfFNVUkJMIC0gTXVsdGk9MHxTRU0tVVJJQkw9MHxTRU0tRlJFU0gxNT0wfFVSSUJMIC0gQmxhY2ssIFVSSUJMIC0gR3JleSwgVVJJQkwgLSBSZWQ9MHxTUEFNSEFVUyAtIERCTD0w
X-SmarterMail-SmartHostSpamWeight: 37
X-SmarterMail-SmartHostSpamSalt: 963260576
X-SmarterMail-SmartHostSpamKey: -1905407280
X-SmarterMail-Spam: Null Sender: 0, SPF [Pass]: 0, DMARC [passed]: 0, Reverse DNS Lookup [ForwardMismatch]: 10, ISpamAssassin [raw:2]: 3, DKIM [Pass]: 0, Custom Rules [BADSUBJECT: 7], BONDEDSENDER: 0, MSRBL: 0, SURBL - SpamCop: 0, MAILSPIKE-WL: -5, SENDERSCORE: 0, SPAMHAUS - ZEN: 0, SORBS: 0, SEM-BL: 0, PROTECTED SKY - Yellow: 0, UBL: 0, MAILSPIKE-Z: 0, BARRACUDA: 0, UCEPROTECT Level 1: 0, SURRIEL: 0, SPAMRATS: 0, MSRBL-Virus: 0, HOSTKARMA - Yellowlist: 0, IADB: 0, HOSTKARMA - Blacklist, HOSTKARMA - Brownlist, HOSTKARMA - Whitelist: 0, WPBL: 0, SEM-BS: 0, PROTECTED SKY - Red: 0, MAILSPIKE: 0, IX: 0, DNSWL: 0, GBUDB: 0, BACKSCATTER: 0, UCEPROTECT Level 2: 0, SURBL - Multi: 0, SEM-URIBL: 0, SEM-FRESH15: 0, URIBL - Black, URIBL - Grey, URIBL - Red: 0, SPAMHAUS - DBL: 0
X-SmarterMail-TotalSpamWeight: 15
X-SmarterMail-SpamAction: Low | NoAction

This example makes it even more clear what is happening. Declude-Scan scores a 22 weight. SmarterMail-Spam scores a 15 weight. Declude-Scan+SmarterMail-Spam = 37 which is indeed reflected in the SmarterMail-SmartHostSpamWeight of 37. However, all that SmarterMail Gateway is passing off to the primary SmarterMail server is the SmarterMail-TotalSpamWeight and not SmarterMail-SmartHostSpamWeight. Maybe these calculations were reversed in recent versions, or the wrong header is being parsed?

Douglas Foster Replied

3/8/2024 at 3:45 PM

This is a design issue, not a big Inbound SMTP checks, spam checks, and outbound SMTP checks are indepenfent processrs. Weight starts at zero in each phase. Declude is part of spam checks phase

Scarab Replied

3/8/2024 at 4:14 PM

@Douglas Foster

If it is a design issue and not a bug then what settings must I change in SmarterMail to get the primary SmarterMail server to read/parse the correct Header value?

The desired value appears to be SmartHostSpamWeight which includes **BOTH** the Declude+SM Spam Checks. I would like the primary SmarterMail server to use this value instead of the SmarterMailTotalWeight which is only including the SM Spam Checks. Otherwise, what is even the point of running Declude (or running any Spam Checks in SM at all!) if you have to allow the lesser of them to pass to the primary SM server?

At this point, if there is no way to pass the correct score from the Gateway to the Primary then I don't see any point in enabling Spam Checking on a Gateway at all (although it cannot be done on the Primary if it was received by an Inbound Gateway). It would appear that "by design" SM cannot be used as an effective tool for preventing Spam, either by itself or with third-party utilities. Or am I somehow wrong? Please explain to me how I can pass the correct score from the Incoming SM Gateway to the Primary SM Server?

Douglas Foster Replied

3/8/2024 at 8:36 PM

Not exactly my expertise, but I did spend some time recently trying to make sense of the SmarterMail spam filtering options.

Correction to my earlier comment: The middle phase is called "Spool Filtering". It is also the only phase that is applied to non-SMTP traffic.

Note that Declude can run in the Spool Filtering phase or the Outbound SMTP phase. In theory, it could run in both phases to total points in both groups, but I think Declude would get confused by that setup.

Communicating results between servers is a little more tricky. I suggest creating unique Declude tests for each result that you want to communicate downstream. These tests results can be communicated downstream with the X-DECLUDE-TESTS: <TestsFailedWithWeights> or with individual WARN actions. (I have been using X-DECLUDE- TESTS.)

On the mail store server, you then create a new Spam Check using "Header" contains <keyword>. Unique keywords are needed because the Header spam check cannot parse the header to extract and use the weight value.

The "Header" test can be applied during Spool Filtering, Outbound SMTP, or both. Since you want it to be applied on incoming messages to your mail store server, it would be configured as a "Spool Filtering" test.

Hope this helps.

Douglas Foster Replied

3/9/2024 at 6:09 AM

So, to try to address the right question

If all of your other tests can be configured for Spool Filtering, then you configure Declude into Spool Filtering.
If all of your other tests can be configured for Outbound SMTP, then you configure Declude into Outbound SMTP along with them.
If either of these options can work, then you get a single total at the Inbound Gateway, and it will trigger an immediate SmarterMail action based on the total.
If the tests cannot all be combined into one section, you may be able to get the desired result using Header tests at the mail store server. That will depend on whether SmarterMail documents the score for each phase in the EML file, which is a question that I have not investigated.

Back to Community Threads

Please leave this box unchecked

7 Replies

Reply to Thread

Tags

Related Knowledge Base Articles