Back to Community Threads
5
Presenting message credibility to the recipient
Idea shared by Douglas Foster - 5/13/2018 at 4:55 AM
Proposed
A first principle of security implementation is that the best defense is well-trained personnel.   Even though email is known to be filled with hostile activity, every user interfaces hides important information from the user, keeping them ignorant.   Several confidence-building features are underused because non-adopters and adopters appear the same to the recipient.   This is a systemic problem, but SmarterMail can at least fix it in their web browser interface, and might be able to provide an Outlook plug-in or ThunderBird plug-in for those user interfaces as well.
 
Below is a list of the information that a mail server like SmarterMail could present to the user to help users understand the characteristics of the current message.

DKIM Signature Status

The From information should be presented with DKIM signature status
  • From: JohnDoe@Example.com (unsigned)
  • From: JohnDoe@Example.com (with invalid signature)
  • From: JohnDoe@Example.com (signed, validated confirmed)
  • From: JohnDoe@Example.com (signed, validity not checked)
This signature status flag could apply to any email address or domain name that appears with any of the material below.

SMTP From Address simplification

Remove VERP, BATV, and SRS formatting overheads out of the SMTP From Address  (formally called Authenticated-As or Envelope-From).   If the local part of the SMTP From contains either the recipient domain or recipient local part, remove these strings as well.   Then compare the simplified SMTP Authenticated-As address to the From header.   If different, report this information.   If no SMTP from information is available, this is significant as well:
  • From: JohnDoe@Example.com
    By login user:  example-com@bigmailservice.com
  • From: JohnDoe@Example.com
    By Anonymous user

List Processing

If the SMTP From contains the recipient domain and recipient localpart as substrings, it is probably from a list.   Similarly if any of the LIST-* headers are present, it is a mailing list.   If the LIST-ID header is present, use that as the list name, otherwise the list ID is unknown.
Then check to see if that list matches either the From Header, the To Header, or the Simplified SMTP From.   This can be factored into the user information presentation.
Original sender address can be inferred from several methods:
  • From header, if different from the list ID
  • Reply-To header
  • SRS encoding in the local part, if present
If the From information was modified by the mailing list, then DKIM signatures might apply to either the mailing list sender or the original sender, so signature status should be applied to both.
This presentation results:
  • If From Header and Reply-To match, or Reply-To is not specified, then the presentation depends whether list-id header is found
    • From: JohnDoe@Example.com (signature info)
      Via mailing list: JohnsFriends@ExampleList.com (signature info)
    • From JohnDoe@Example.com (signature info)
      Via unknown mailing list
  • If From Header matches the mailing list id, and Reply-To is present, then the reply to takes precedence:
    • From: JohnsFriends@ExampleList.com (signature info)
      Reply To:  JohnDoe@Example.com (signature info)
  • If From Header matches the mailing list id and Reply-To is not present, but SRS coding is detected, then the apparent sender is extracted from the SRS encoding:
    • From: JohnsFriends@ExampleList.com (mailing list) (signature info)
      Apparently Submitted by:   JohnDoe@Example.com (signature info)
If multiple List-* headers are present, there should be a button for [List Details] which presents all of the List-* headers, the reply-to, and the return-path.

Forwarded Mail

Forwarded mail is indicated (a) if the “For” token of the Received-From header changes, or (b) SRS encoding is present.  If the For information changes, it is desirable to determine the identity of the server that forwarded the message.   But since this is probably a server on a private IP address, the host name and address are not useful.  I would suggest using the first server after the name change that is not associated with a private IP address.   If a suitable identity cannot be determined or is to difficult to identify, the server information could just be omitted.
FOR address change provides a full address as the forwarder
  • To: JohnDoe@Example.com
    Forwarded from:  PriorAddress@Someplace.com
    By server:  hostname.someplace.com
SRS encoding only indicates the domain that takes responsibility for the message, and the server where this occurred cannot be reliably determined by any known method:
  • To: JohnDoe@Example.com
    Forwarded from:  @Someplace.com

Received-From Information

Received from information provides useful information about the message flow.   Although it can be spoofed, this is not often observed.  Viewing the raw headers is unknown to most users, and is a bit obscure to even knowledgeable users.
Provide a button for [Message Flow Details] which presents the Received-From history in tabular form.
  • Sequence number, host name, host IP, Encrypted hop (yes/no)
If may be desirable to suppress private IP hosts by default, with an option to display them as well if more details are desired.

Authentication-Results

RFC 7601 provides a header for a perimeter spam filter to communicate results information to a mail server or user interface downstream from it.   If present and trustworthy, these results should be available to the user as well.
If the mail server or post office performs these tests locally, then the same information can be displayed:
  • Authentication:   Received from a server authorized by SPF to send for @Example.com

Multiple Domains

There are many header fields that have a value in email address format.   When a lot of different domain names are present, suspicion arises.   Some of these have already been presented in previous sections, but some others are relevant as well.  Return-Path is the first one that comes to mind, but there may be others as well.
  • If Return-Path is different from the From header, SMTP From, or simplified SMTP from, then the return-path should be reported
    Return Errors To:   postmaster@Example.com
  • Message ID problems:
    • Message ID:  Not found
    • Message ID:  Does not contain a valid domain name
    • Message ID:  Indicates sending domain is @Example2.com, not @Example.com
  • This message contains headers referencing these additional domains: domain1, domain2, ...
 

4 Replies

Reply to Thread
1
Douglas has some good ideas here that could also be folded into 5 or 6 new spam tests to use in the fight against spam.
Kendra Support http://www.kendra.com support@kendra.com 425-397-7911 Junk Email filtered ISP
0
Thank you, Matthew, for your encouragement.

To be clear, I don't think anything in my list involves proof of spam status. I am just tired of the pretense that spam filters are 100% perfect. Because of that pretense, we am supposed to assume that if an email arrives in any inbox, we don't need to know anything about the message's characteristics because the message is certainly worth our time and worthy of our trust.

Some advertising messages are useful, but it is still very useful for me to know whether an email message is a one-to-one communication from a friend or a marketing message generated by a machine.

Some of the characteristics in my list may be useful to a spam scoring algorithm. I actually assume that most of them already are included in the available spam scoring products. But in the end, it is the end user who decides whether to read a message, decides whether to reply to a message, and decides whether to click on links in those messages. Why should we waste his valuable brain by keeping secrets from him?
0
Douglas,
As far as I can tell only DKIM is directly covered by the Spam filters. I for one would like some more options in the SPAM fight. Sometimes I think my customers don't want to be bothered and they also don't want any spam.
Kendra Support http://www.kendra.com support@kendra.com 425-397-7911 Junk Email filtered ISP
3
Douglas,
 
Good ideas!  Thanks for sharing. It would be nice to present more information to help the user decide whether/not to trust a message.
 
And granted, there is not a definitive test for spam, but as Matthew suggested, some of these ideas could be rolled into SmarterMail's spam scoring system to get more accurate classifications.
 
Also, some of these ideas have ongoing threads. Here's a post with links to multiple spam filtering and anti-spoofing suggestions:
 
Kevin
Back to Community Threads

Reply to Thread

Related Knowledge Base Articles

Common Email Delivery ErrorsSmarterMail > Troubleshooting
Use LDAP with OutlookSmarterMail > Desktop and Mobile Synchronization
Copy Mail Sent To or From a MailboxSmarterMail > Domain/User Configuration and Management
Deploying Rspamd For Use With SmarterMailSmarterMail > Server Configuration and Management
Import/Export a PST File Using OutlookSmarterMail > Desktop and Mobile Synchronization