I downloaded my spam config and reset it and then tweaked it barely. We'll see how that works out. Here is what my prior config was which worked fine up until 6970.

Check the SpamRats and Surriel responses and see what they are set to, I believe 6970 changed them to *

Not sure why though, The PSBL response is 127.0.0.2. Same with SpamRats... Dyna is 127.0.0.36, NoPTR is 127.0.0.37 and Spam is 127.0.0.38. I would think saying any response (*) is bad. As if the server returns say 127.0.0.1 or say some 127.0.0.255 for over query limit, now all your emails are failing.

After resetting my spam settings to the default (after downloading a backup) and tweaking a few things such as enabling grey listing, it's working well now.

Processed a return for an amazon item. Never had an issue with it flagging as spam in SM 16 and prior. Using default settings in 6970 after the reset. Amazon return email got flagged as spam due to a score of 18. Something's not right! I flag spam at weight of 15.

I can confirm that all rules that use a * fail. 

 Not good if you have a set spam over 30 to be deleted.

Cheers

Neil

The DKIM_Fail is a concern, does the WatchGuard device modify the email invalidating it's DKIM signature? Do you have other emails with a DKIM signature that pass?

DKIM has never been an issue, Watchguard is not affecting it. It does have its own anti-spam system but higher thresholds for confirmed spam only. This started with 6970. Other emails are coming in, I'll have to check the DKIM stats on those. 

If you paste the unmodified raw content from the Amazon email into http://www.appmaildev.com/en/dkimfile does it pass the DKIM test?

We're not yet using v100 but I see in the changelogs:

6970:

6964:

In your mail headers I can see reference to SpamRats and Surriel. Maybe one of these fix/changes made these RBL matches when they shouldn't ? Just a guess... Maybe you should check manually your mail against these list to see if they really matches or if it's a bug.

Spamrats should be;
Weight                          5
Hostname                        spam.spamrats.com
Required Lookup Value           127.0.0.38
Enable Spool Filtering          Enabled
Enable Incoming SMTP Blocking   Enabled

Here is the spamrats info page. I really hope ST has taken the time to ensure each and every entry they include is accurate and up to date. I hope they will, if needed, validate all entries to ensure they are in fact correct.

I also suggest a "checkup" button that checks each of the entries knowing if one is modified or default and list any items that are not matched to the latest defaults shipped with the latest build and offer to reset.

An example of this is using forum'ware where someone modifies the template (I use vBulletin). The template knows it's modified and offers a reset option. You can also get a list of all modified templates in the system. Apply this logic to spam checks, ST can show which are not matching with the defaults and offer a reset button on each as needed. This would help us down the road update an entry that has been changed by ST instead of going years after the initial install with something incorrect.

There have been a few configuration issues around the Antispam checks that have been highlighted to ST recently and they are working on a knowledge base article to try and address some of these issues and correct the settings in a near future release. The new settings are based on those found here https://github.com/SteveUnderScoreN/SMSpamConfig , there's a PowerShell script that can bring in the latest settings along with the corrections if you want to import them. I could create another script that would compare your current settings with those recommendations and advise, I'll put it on the list.

The questions around DKIM are still a concern.

I can also confirm that all rules that use a * fail.  

I am on version 6970.

Use the following;

Spamrats should be;
Weight                          5
Hostname                        spam.spamrats.com
Required Lookup Value           127.0.0.38
Enable Spool Filtering          Enabled
Enable Incoming SMTP Blocking   Enabled

Surriel should be;
Weight                          5
Hostname                        psbl.surriel.com
Required Lookup Value           127.0.0.2
Enable Spool Filtering          Enabled
Enable Incoming SMTP Blocking   Enabled

We too are seeing this issue - updated to Build 6970 and now SPAM scores have basically doubled. The ones that always give high spam scores now are:

MCAFEE

SEM-URIBL

@ Ionel Aurelian Rau 

Are these set to * for the required lookup value?

Yes, * is set as the required lookup value.

I`ve disabled these 2 checks as they were causing us to miss emails. I did not change anything for them - I`ll just leave them disabled until we hear what`s the problem.

Hmm, even after disabling them, they still appear run checks and assign scores. I restarted now the SM service - is there anything else that needs to be done apart from disabling the "Enable Spool Filtering" toggle?

@ Ionel Aurelian Rau 

Set SEM-URIBL to lookup 127.0.0.2 and set MCAFEE to lookup 127.0.0.3

Did you have them disabled for both incoming SMTP and spool filtering but still got failures?

Incoming SMTP was disabled by default for both - just Enable spool filtering was ON. I disabled that and thought I called it a day - but email was still filtered by these 2 and a lot of legitimate mails landed in the Junk folder. So then I also disabled the toggle from Required Lookup Values (not sure if this is supposed to stay ON or not; I just wanted to have everything disabled) and restarted the SmarterMail service. Then the block lists stopped filtering email.

For now I will not reconfigure or re-enable the 2 blocklists as I do not have the time to deal with any more potential issues. Let`s wait for an official update from SM Tools.

Also, if a service restart is needed to make sure antispam changes take effect, we should be warned of that somewhere in the interface.

Email from Apple just flagged as spam. ST you need to get this spam stuff nailed down. No reason for a regression this many years into your product. We keep going backwards with your updates/upgrades.

Is using Spamassassin really effective anymore? Especially since there is no training for it.

Till build 6964 no issue on spam control. After we upgrade build 6970 all spam weight drastically increase in particular incoming mail.  Even we tried default setting recommended by smartermail 

Please have a look at msn spam weight 

Return-Path: <mail@msn.com>
Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-oln040092003047.outbound.protection.outlook.com [40.92.3.47]) by mail.domain.com with SMTP;
Tue, 5 Feb 2019 05:41:51 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=msn.com; s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=jwkPYz4EiFSiQ0K2iHV5jOwpubQDpueA+ohOrYtfMZM=;
b=nXEDXoRxtQQfRqaFHvvs+WoUm2NCkWfU1O/gLo89l+r3WXJeTh8KvzROFFyNkCOnLqJUrCqO2B7F3eRvgkp6yE6iOHniZCn9BS8DBIWnqAQP2uTUM2bvHffyh+1bbE1fPhFQeca1CB7HiToJEqZxr/1H4ViYLitPbGYZ2FZeFfsIYSkMTIKVLl1ZhdcDgY8Qy6a64YHpcoj/hignR2v16i/IZe3brNTGV61nee634AltiUCHB1umFEvhgKAhd5Vrp8KkEYl8Ag1mBVKkSUilJA7C0dmceyKYKYVxyiGpW+lWOPK6nTLJ4XogHR6/KOy8A0emhpxjGLomrCB1vV8NBA==
Received: from BL2NAM02FT060.eop-nam02.prod.protection.outlook.com
(10.152.76.59) by BL2NAM02HT153.eop-nam02.prod.protection.outlook.com
(10.152.76.215) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1580.10; Tue, 5 Feb
2019 10:41:47 +0000
Received: from BYAPR20MB2551.namprd20.prod.outlook.com (10.152.76.53) by
BL2NAM02FT060.mail.protection.outlook.com (10.152.76.124) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.1580.10 via Frontend Transport; Tue, 5 Feb 2019 10:41:47 +0000
Received: from BYAPR20MB2551.namprd20.prod.outlook.com
([fe80::a944:594f:a013:184d]) by BYAPR20MB2551.namprd20.prod.outlook.com
([fe80::a944:594f:a013:184d%4]) with mapi id 15.20.1580.019; Tue, 5 Feb 2019
10:41:46 +0000
From: Binesh Shamunni <mail@msn.com>
To: "binesh@domain.com" <binesh@domain.com>
Subject: Re: Ok
Thread-Topic: Ok
Thread-Index: AQHUvFZ5e3vdkkB9t0SVU2LlCCR/VqXPNIiAgAHRk+8=
Date: Tue, 5 Feb 2019 10:41:46 +0000
Message-ID: <BYAPR20MB25517C93B392E0234A42B60DC56E0@BYAPR20MB2551.namprd20.prod.outlook.com>
References: <BYAPR20MB255168F8C99AEEACEDADE092C56D0@BYAPR20MB2551.namprd20.prod.outlook.com>,<711229b440fa43329d8b35e7761bcc7e@arieltechnologies.com>
In-Reply-To: <711229b440fa43329d8b35e7761bcc7e@domain.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-incomingtopheadermarker: OriginalChecksum:5867759B2EECD8F6B6FD19D2533CCF42BE7EC793B13F2EDDF33B07759B9E0F06;UpperCasedChecksum:295FD30FEEDBC936D7CFEA400493EF385B09EC1ED3F9385FFFE426CD95324C72;SizeAsReceived:6933;Count:45
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [ciKE89ya3utqKNNllJ7Ro7gqTWCMz7iT]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;BL2NAM02HT153;6:68EGErkggjQq9FYIhoBUOSuHvpQDPOEyzumtw3HnrfV952yPuHaDFmU/DGqn3nX3XJGSv9e6ka0+EponKHag+B7Qp99WTAoqLkJJdN1tq/ptkZhkV3VHdV3II3QGThxoDs4X0LREiA4Y9AKzoTHdgAl5F5hLC4+NCRMCTPkPcNcN/fLe4grZ2GTLw4IMTYS6dNXy1U5Uz4CS336cQM1N8KG9xqVaXcNPJ2MTwvbAOMC3V0IJ5WNfGSweUEcugutI//qWSkbu0XNMl9SPrUXXVjYjbt31pEzTI2HL7ZRsuRdeUfkwF7LPN3hdzcThM0b4rxYOF7v+vdIqiY8WVxCjsxh9foAUc4v+Ql4ZvtM8JHgyGBqjEA/PfgrkqavVotKEY7rZUizc3/oZd7pxFK8XVY9+WKGsoW94hc5GkJ8QkUJ8z2bJFljtaiItztKum9QK8ZIOouGrWVlx/wSGRbB2gw==;5:YxBrCT2s3/vAEAa6F5xg6/qw3eYY7KM2k6ED1JjS3Re2hALBrrsyB8aMfolsVaE47JzqLZk0c7P1oqPg4sxzU40AUOcXMP90+sYor9gbKPfQqdMXTW5W1B28rOxA58wuZeFcGBp9sWiFgGX56v+h3Fh7/4fZGuBpIDpL3ItWmHavKSnNNt4dLr+yIeluAAItGvB/L8P7OOwfHk+GiuL7WQ==;7:Z998N+d9jiz+z+zean7PhNaqNOgM/9hRHIoE0dw0twgRqSg8pt5FbFOnr7MtMUfhP5inanpl0DOiB0NRfN9vOIjxtszhytAS0tyMfv4h8kq/iqBDE0yYfW96XhYr7zfvoJo8wZR+wqA4avwbQL+OTw==
x-incomingheadercount: 45
x-eopattributedmessage: 0
x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031323274)(2017031324274)(2017031322404)(1601125500)(1603101475)(1701031045);SRVR:BL2NAM02HT153;
x-ms-traffictypediagnostic: BL2NAM02HT153:
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(4566010)(82015058);SRVR:BL2NAM02HT153;BCL:0;PCL:0;RULEID:;SRVR:BL2NAM02HT153;
x-microsoft-antispam-message-info: l9m3s5wQ54ZKcR+tTj84WgOLKGHoyPk2m2rwDG/X931QWnUFot2syQ9ISUe/kBA4
Content-Type: multipart/alternative;
boundary="_000_BYAPR20MB25517C93B392E0234A42B60DC56E0BYAPR20MB2551namp_"
MIME-Version: 1.0
X-OriginatorOrg: outlook.com
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: c001924d-3e68-4f40-89c2-901a49278da7
X-MS-Exchange-CrossTenant-Network-Message-Id: d4d8f7b5-1060-4313-1ce3-08d68b5686db
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: c001924d-3e68-4f40-89c2-901a49278da7
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Feb 2019 10:41:46.5008
(UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2NAM02HT153
X-CTCH-RefId: str=0001.0A020209.5C596874.0036,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0
X-CTCH-AVLevel: Unknown
X-SmarterMail-Spam: Reverse DNS Lookup [Passed], SPF_Pass, SPAMHAUS - ZEN, SPAMRATS, SURRIEL, UCE PROTECT LEVEL 2, UCE PROTECT LEVEL 3, Cyren 0 [value: Unknown], ISpamAssassin 0 [raw: 0], DK_None, DKIM_Pass, SEM-URIBL:2, SEM-URIRED:2
X-SmarterMail-TotalSpamWeight: 54

@ Neal Culiner

Did you set the Spamrats settings as per my recommendation?

Spamrats should be;
Weight                          5
Hostname                        spam.spamrats.com
Required Lookup Value           127.0.0.38
Enable Spool Filtering          Enabled
Enable Incoming SMTP Blocking   Enabled

Also with the DKIM issues in 6970 I suggest you disable DKIM checks for now.

@ Binesh Shammunni

Are some of your checks looking for * as the lookup value?

@Steve Norton 

let me know control details ie  * as lookup value ? so that I can correct the value

This is an example, presumably, where ST fails in their development and test cycles. They make a change and immediately push it to production so we test it at the same time they do. When they make changes to anti-spam for example they should test it and use it on their production servers at LEAST one week prior to it going to the public. They would see the spam scoring issues before we did and fix it prior to it ever getting to us. But instead they tweak, release, triage, tweak, release and we are in a constant test cycle for them where we they should have extensively used the changes before it ever getting to us.

@ Neal Culiner

Just putting that aside for a minute, have you changed the Spamrats settings and disabled DKIM, yes or no?

@ Binesh Shammunni

What rules have * as the lookup value, I need the URL from the rule and he rule name.

Steve, I'm not making any changes. I'm waiting for a correction from ST and then I'll reset again if they don't have a system to update invalid entries automatically. 

Steve Norton,

I used your values and it solved the issue for me.

What I have found is that any with a Required Lookup as Value as '*' fails. Those with a 127.x.x.x seem to work.

Curiously disabling the lookup value also had them returning a pass, so I can only think that

6970:

has had the impact

Many thanks

Neil

Okay, I went ahead and disabled DKIM and updated spamrats, will see how this works.

Sure would be nice if there was a checkbox select then enable/disable feature on the row.

I'm concerned that changing a spam setting may not take immediate affect as there is no save button and therefore SM cannot detect when it's updated. If SM is caching the spam config, which they probably are for performance, how is it getting cleared and updated any time a spam setting is updated? ST care to answer? 

Therefore I can only trust that after changing any spam setting the service should be restarted. I'd like to hear the technical details on this and if changing a setting is reflected immediately in SM. 

Hi all,

I wanted to provide more information regarding the wildcard for the RBL lookup values. In Build 6970, we resolved an issue where the wildcard (*) for an RBL's Required Lookup Value was not working properly. In this build, if an RBL is configured to use * for the lookup value, ANYTHING that is returned from that hostname will trigger the corresponding weight to be applied. Please take note that some RBLs return for "good" emails. Therefore, if there are RBLs on the same hostname that return both good and bad results, the wildcard should NOT be used, since the wildcard does not differentiate between pass and fail. In those cases, you'll need to use individual lookup values instead.

Since this release, however, it was decided that it would be best that SpamRats use a lookup value of 127.0.0.38. Therefore, in the next release, we've modified the default configuration for the SpamRats RBL, changing the Required Lookup Value from the wildcard (*) to 127.0.0.38. 

In addition, we found that ANY return from the hostname was causing the spam weight to be applied, even if the return was 'host not found'. We've fixed this issue in the next release as well. 

Thank you,

Hi Neal, 

In response to:

Restarting the SmarterMail service after modifying your antispam configuration should not be necessary. Changes you make in the Antispam section are immediately reflected in the settings.json file. However, there may be some caching occurring with the spam settings page, and we're currently looking into it.

Andrea, I ran a test and proved just changing the settings is not immediate. Per the support ticket you sent me I stopped Watchguard from adding headers to see if it's causing the DKIM fail. So I applied that change to my firewall then went into SM to re-enable DKIM. I did a test from gmail to my SM server. No mention of DKIM. I restarted the SM service and repeated the test, now it shows the DKIM result. So this confirms to me that the anti-spam config change does not take affect right away and who knows how long it is cached, FYI.

I reported this issue as well through tickets. I added a URIBL lookup and the lookup didnt take effect until after I restarted SM.

Please provide custom build we have huge issue on production environment.  

Thanks

So we have bad anti-spam settings in 6970. A new build will be installed, I assume spam settings are not overwritten when a new build is installed? ST, please explain how you all update anti-spam settings for prior installs. Do we need to download the config, revert/reset our settings? 

Please outline the proper procedure when ST updates anti-spam settings. It sounds like anytime there is an anti-spam setting change the change log must clearly indicate what was revised so manual revisions can be made. But the problem is someone doesn't upgrade for a while, they aren't going to review each release notes to figure out what needs to be set. This again goes back to my suggestion of each entry needs an indicator if it's different from the default so we know what's modified. But for now, what is the proper process to ensure each user is using the correct/latest settings?

Hi Neal, 

This concern was recently brought up in the Community, and as a result, we're putting together a document that outlines the default configuration changes that are made with the release. This will allow those with existing installations to know the changes they need to make after upgrading. 

Ron, we all share that sentiment and had hoped since the major incident a few months back of pushing a bad build prior to a long weekend was over. ST has to take on an enterprise mentality vs. a home small biz approach. They need to think of their builds being like Microsoft Windows Updates going out to millions. Do you tweak and release or do you extensively test including own use prior to releases going to public. Until ST can take on an enterprise approach to software development this will continue. Fortunately they are receptive, sadly the same problems continue which only destroy business for them. Someday we'll all just use gmail!

Thanks Matt Petty for providing custom build for fixing spam issue 

Lot of bugs still persist in custom build as well,  stable version like 15.7 or 16.3 worked well same setup.

The server and domain implemented https access , google chrome browser tested below issue

1)  active sync not working well , 

2) calendar appointment not working well , there is still a lag when entering info to set an appointment. 

3 ) mail draft not working well manually save duplicate entry,  

4)  spool not cleaned and not correct notifications , 

5) long delay moving page in admin part, 

6) google chrome keep disabled notification , we have enabled notification & pop ups manually, its automatically disabled
7) Mail alerts still on manual process after implementing https mode for both server and domains.

Customer are screaming lot of bugs on new environment   even myself lost trust more than 13 years associate with smartermail as a user/admin

Thanks