8
Biggest Customer Complaint - Trusted Sender
Question asked by John Marx - 11/16/2022 at 9:11 AM
Answered
The biggest complaint every customer raises is they use webmail, they mark someone in junk as a trusted sender, and they continue to go to junk mail. How can I get trusted sender to actually be trusted and stay in the inbox? 

If they right-click on ANY, even after marking, the emails say trusted sender so obviously SmarterMail is smart enough to know that. 

19 Replies

Reply to Thread
1
Jay Dubb Replied
We've found that the actual "from" address can be different from that shown in the message.  For example, a message might show as "From" bobjones@company.com  but if you look the headers, you might see bobjones-compan-com-202987345@bounces.mailservice.com as the actual sender.  

If you use Declude, that makes it really easy to find the actual sender.  Just look for the "X-Declude-Sender" header near the bottom.
 
0
John Marx Replied
The headers are always the same. Either way if they were not if when you right-click and the system already says trusted sender it should be smart enough to keep it in the inbox (or really run any rules, if there are any -- each client has no rules so they should just go to the inbox).
1
Tony Scholz Replied
Employee Post Marked As Answer
Hello John, 

Trusted Senders is designed to allow you to skip SPAM checks for specified email address and domains. This is skipped/ignored in 2 cases. DKIM and SPF are still checked to protect your system, 

Trusted Senders
Domain Administrators can add specific email addresses (such as jsmith@example.com) or domains (such as example.com) that will be exempted from spam filtering. This can prevent mail from friends, business associates and mailing lists from being blocked and lets the system know that these messages come from a trusted source. NoteEmail addresses in a user's contacts are always considered trusted senders. In addition, if users unmark a message as spam, the sender is automatically included on their personal trusted senders list.
Here is an article that has a more robust description of why SPF and DKIM are exempt from the Trusted Senders. 
When the DKIM and SPF both pass the SPAM weight is zeroed out for "Trusted Senders". The header for this looks like this. 

X-SmarterMail-Spam: SPF_Pass, HostKarma - Whitelist, Reverse DNS Lookup [Passed], ISpamAssassin 0 [raw: 0], DK_Pass, DKIM_Pass
X-SmarterMail-TotalSpamWeight: 0 (Trusted Sender - Domain)
When either the SPF and/or DKIM fail then the full spam weight of all failed checks are brought forward and passed on. This header will have a line like this. 

X-SmarterMail-Spam: SPF_SoftFail, Reverse DNS Lookup [Passed], ISpamAssassin 7 [raw: 5], DK_None, DKIM_None
X-SmarterMail-TotalSpamWeight: 19 (Trusted Sender - User, failed SPF)
The other reason that this may be happening is what Jay mentioned above. Take a look at the raw content of the message and look at the "Return-Path:" and see if it matches the "From:" field. When you mark an email as a trusted sender it is grabbing hte email address in the From: field, when this check is processed during the SMTP session it looks at the Mail From: address that can be adjusted by the sender in the DATA portion of the SMTP session. 

For Example:

In the above example notifications will be the trusted sender and pm_bounces is what will be checked during the SMTP session. 

I hope this helps. Thank you
Tony
Tony Scholz System/Network Administrator SmarterTools Inc. www.smartertools.com
1
John Marx Replied
The problem I am seeing is many email servers now use SMTP gateways. These gateways allow reporting of spam, bounces, etc. They manipulate the message envelopes causing trusted senders to not be trusted. Here is an example of one of my trusted senders on my account that seems to "love" the junk mail folder based on your picture.

Return-Path: <bounces+8754967-3ee2-john=fawkesdm.com@ckespa.neilpatel.com>
From: Neil Patel <np@neilpatel.com>

10
Jay Dubb Replied
ENHANCEMENT REQUEST:  Make the Trusted Senders function more intelligent, to read all of the headers involved in the "trusted" decision.  

We have this exact same complaint from our own customers, many of whom receive extremely time-sensitive documents via email (real estate, insurance, title company, ...) and they are frequently complaining they've marked mail as Trusted, only to have it continue going to Spam.

We've had to work (way too many times) with their I.T. team to search the headers and built a domain-wide filtering rule to whitelist the "actual" senders of these messages-- something we wouldn't have to do if the "mark as Trusted Sender" function worked in the manner users would "expect" it to work.
 
2
+1

Gabriele Maoret - Head of SysAdmins at SERSIS Currently manages 6 SmarterMail installations (1 in the cloud for SERSIS which provides services to a few hundred third-party email domains + 5 on-premise for customers who prefer to have their mail server in-house)
2
Jaime Alvarez Replied
+1
0
Paul Blank Replied
+!
0
YS Tech Replied
I have many emails now being moved to my spam folder, with no way of setting them to be trusted senders (as that functionality doesn't really do what it says).
Having the feature that does recognise the email as an actual trusted sender and allows it through would be really handy as I can't tell my clients any way to trust an email or even a domain as it just doesn't work.

e.g. one example sent to me, they obviously use a third party klaviyo system and a different domain to the business domain. I have added this domain to the trusted senders and SM obviously recognises it but still moves it to spam.

From: Pro Bike Tool "><probiketools@razorgroup.com>
...
X-Declude-Sender: bounces+32138063-631b-comps=*******.net@send.ksd1.klaviyomail.com [149.72.88.115]
X-Declude-Spoolname: 315424743.eml
X-Declude-RefID: 
X-Declude-Note: Scanned by Declude 4.12.11
X-Declude-Scan: Incoming Score [19] at 16:32:11 on 22 Sep 2023
X-Declude-Tests: HOSTKARMA-BLACK [10], MAILSPIKE-H4 [-4], SORBS [4], SORBS-NEW [3], SORBS-RECENT [3], SPFPASS [-1], FROMNOMATCH [2], HAM-INDICATOR [-2], FILTER-BULK [4], WEIGHT10 [10], WEIGHT14 [14]
X-Country-Chain: UNITED STATES->destination
X-Declude-Code: e
X-HELO: o1408.shared.klaviyomail.com
X-Identity: 149.72.88.115 | o1408.shared.klaviyomail.com | send.ksd1.klaviyomail.com
X-SmarterMail-Spam: DMARC [skipped - DMARC Disabled]: 0, Reverse DNS Lookup [Passed]: 0, SPF [Pass]: 0, DKIM [Fail]: 5, Declude: 19
X-SmarterMail-FoundTracker: sendgrid | SendGrid
X-SmarterMail-TotalSpamWeight: 24 (Trusted Sender - Domain, DMARC: Skipped (DMARC Disabled))
X-SmarterMail-SpamAction: Medium | MoveToFolder
muc-off.com is another domain that uses klaviyo

This one is an online accounting domain, also gets moved to spam:

From: QuickFile "><quickfile-noreply@quickfile.co.uk>
...
X-Declude-Note: Scanned by Declude 4.12.11
X-Declude-Scan: Incoming Score [23] at 02:10:30 on 22 Sep 2023
X-Declude-Tests: MAILSPIKE-H5 [-7], SORBS-RECENT [3], NOABUSE [2], SPFPASS [-1], FROMNOMATCH [2], HAM-INDICATOR [-2], FILTER-BULK [4], FILTER-SPAM [20], FILTER-SUBJECT [2], WEIGHT10 [10], WEIGHT14 [14], WEIGHT20 [20]
X-Country-Chain: UNITED STATES->destination
X-Declude-Code: e
X-HELO: a8-73.smtp-out.amazonses.com
X-Identity: 54.240.8.73 | a8-73.smtp-out.amazonses.com | amazonses.com
X-SmarterMail-Spam: DMARC [skipped - DMARC Disabled]: 0, Reverse DNS Lookup [Passed]: 0, SPF [Pass]: 0, DKIM [Fail]: 5, Declude: 23
X-SmarterMail-FoundTracker: mailgun | Mailgun
X-SmarterMail-TotalSpamWeight: 28 (Trusted Sender - System, DMARC: Skipped (DMARC Disabled))
X-SmarterMail-SpamAction: Medium | MoveToFolder
Is it that we need to trust klaviyomail.com and amazonses.com for these to come through, if so that would open us up to a lot more issues wouldn't it?

Thanks
0
Millennium Systems Replied
YS Tech

If DKIM fails like both your examples, the trusted sender status is ignored.

1
Paul Blank Replied
It's just sad that here in the 3rd decade of the 21st Century, email admins can't have SPF and/or DKIM configured at the least (let alone DMARC). 

Nonetheless there should be an easy way to trust these senders on the SM end without the hassle of creating a filtering rule, especially if you can verify by the sending servers' IP address(es). Good Luck!
2
YS Tech Replied
Yes, I know. There are lots of variables and the DKIM failing being one (which could be construed as not a valid sender) but if I trust a sender address, i'd like the sender address to be trusted, whether DKIM is correct or not.
There are so many emails that fail this setting that it makes the trusted sender option in SM useless.
If SM had a checkbox under the "Trusted Sender" option that was "Ignore DKIM", "Ignore SPF", "Ignore DMARC" as well as just Trusted Sender, that would help considerably. Or even separate lists for each of those ignores?

1
YS Tech Replied
Unfortunately, clients would like to be able to select a specific email address as a trusted sender and have it then come through. They'd rather it be the odd dodgy one that comes through, than not get the valid ones at all.
Even if it was appended to the subject *Failed SPF* or something like that, it would be better than not getting them at all (or having to sit there sifting through the spam folder).

Quite a few of these emails are from accounting software or mailing lists clients have signed up for and most are DKIM fails. It's quite obvious that these valid emails are not set up correctly on the sending server, but the client still wants them and currently, there's no way to give the client that email without them manually sifting through all the other spam in the spam folder.
I already have a couple of clients who just want to receive everything without filters, so they don't miss any valid emails. If a client misses 1 valid email it can cost them thousands in some cases, hence the requests.
1
Douglas Foster Replied
You forget that the Internet is not a place filled with nice people.   It is a dark alley where nice people go at their own risk, often to their peril.   If your spam filtering is doing its job, more than half of your email gets blocked.  That means that if you blocked all traffic, you would be right more often than not!

The worst possible scenario is to accept a fraudulent message identifier as legitimate, and then exempt the attack from all content filtering because it has been falsely labeled as "trusted sender".   Fraudulent email does not carry a warning label, so the only way to protect yourself from this scenario is to require that whitelisted senders are verified.   Unfortunately, we do not have a standards-based technique for obtaining a verification result on every message.

IETF has defined two ways to validate two different parts of the email message:   

SPF verifies the SMTP Sender (a.k.a. Mail From) address [RFC 5321].   Some senders don't have an SPF policy at al.   Others have configuration mistakes, so that messages don't produce PASS even when they would if configured correctly.   SPF does not have a feedback mechanism, so the errors tend to go unresolved because they are never reported.  If forwarding has happened, SPF because either useless (Mail From is rewritten to match forwarder domain) or misleading (SPF fails if Mail From is not rewritten). 

We don't have a very good way to identify forwarded messages, and the one tool that we do have (ARC) is wholly depending on the forwarder providing correct information, and non-forwarders not supplying data that will by definition be misleading.

DMARC verifies the Message's "From" header [RFC 5322] using either aligned SPF PASS or aligned DKIM PASS.   Alignment can be "strict" (exact domain match) or "relaxed' (same organization).   Alignment is determined by the DMARC policy, which is part of the reason that DMARC is only defined when a policy is present.    One could note that exact match on SPF or DKIM does not depend on the existence of a DMARC policy.    I have made that observation but it will not be in the next revision of the DMARC specification, because IETF is only interested in protecting the receiver if the sending domain owner wants to provide this protection.   On the upside, DMARC policies are difficult to get completely wrong, so if they exist they should be usable.  However, some domain owners publish p=reject when their messages don't have DKIM signatures.   if the message is forwarded, the message fails authentication and is labeled as a fraud.  If a message is forwarded with changes (common action by mailing lists and some spam filters), any DKIM signatures are invalidated, so the result is again FAIL.  Also, a lot of messages do not have DMARC policies.

Mail From and From addresses are often different   At lot of mail, my guess is about 50%, is sent by Email Service Providers (ESP) who specialize in delivery of messages in high volume and messages generated by web sites or other automation.   The ESP domain is used for the SMTP Mail From address, and the client address is used for the message From address.   There are other reasons as well.   This is not fraud, this is just the way people need email to work.

Overall, both SPF and DMARC return results of "PASS=verified", "FAIL=repudiated", "Uncertain", and "No policy".   Uncertain and No Policy are all too common, and false FAIL will also occur.   

In the example you provided, it appears that the message was going to Junk because the message was producing FAIL.   So making it a trusted sender does not solve the problem because the sender has an inherent trust problem.

In my environment, I have defined a custom exception structure so that I can configure alternate authentication for any wanted message that cannot produce PASS.   It has required a lot of data collection and a lot of message review.   But along the way I have detected and blocked a lot of malicious and unwanted senders, not merely impersonators, so I feel like the effort has been well worth it.   But this goes way beyond what you can expect to get for free from a vendor whose primary business is mail handling, not spam filtering.  I don't think you can even buy it.  But you could build it.

Doug Foster



0
YS Tech Replied
Thanks Doug for your very in depth response.
Great info there and I seem to be going through the same process you have of understanding how things are working and trying to find a workaround.
Completely understand what you're saying about not letting all email that says its from the address you want to come in, as that's easily spoofed.
I'll just have to figure out, as you have, a way of handling the mail differently.
0
Jay Altemoos Replied
Just to chime in on this thread, the SPF lookups in SM 8629 are labelling some emails that have valid SPF records published as failed when they should not be. I opened a ticket with SM support and they did identify an issue that needs development's attention to resolve. Trusted senders are subjected to SPF lookups along with DKIM checks. I would recommend spot checking some of the trusted sender emails that got sent to Junk email for SPF failures and verify that against MXToolbox.com. When I upgraded to 8629 from version 15.7 I noticed a large amount of emails getting thrown into Junk email when they should not have been. What I did in my settings was score SPF failures and DKIM failures to a lower score so not everything was sent to Junk email until development implements a fix. It's not perfect, but it did calm down the amount of email showing up in Junk Email.
2
Jay Altemoos Replied
Hi Ron, I noticed that as well. Hopefully they fix it soon.
0
Paul Blank Replied
I have seen SPF failures - at the receiving end - by other mail servers besides SM. Guessing that when trying to read the (valid) SPF record in the zone file, it's timing out.
1
Jay Altemoos Replied
Hi Paul,
I don't doubt that other mail servers have a similar issue, after our upgrade from 15.7 to 8629 the massive influx of SPF and ReverseDNS failures was quite noticeable. I fully understand that nothing is perfect but 15.7 didn't seem to have as big of a problem with this at least from what I recall. I just hope there is a fix to calm this down on the horizon.

Reply to Thread