5
SMTP and DKIM (build 8818)
Question asked by Sébastien Riccio - 2/29/2024 at 11:03 AM
Answered
Hello,

Today we discovered that mails sent through SMTP on our SmarterMail installation fails the DKIM signing verification, while when the mail is sent from the webmail or EWS, the DKIM signing verification pass.

I tested with multiple mailboxes / domains and with different mail clients configured with SMTP (emClient, Thunderbird, etc)

Anyone else noticed this very critical issue ?

Kind regards,
Sébastien

Sébastien Riccio
System & Network Admin

32 Replies

Reply to Thread
1
Roger Replied
Hello

That's right, I tested it and it doesn't seem to be valid for me either with various checks like mxtoolbox etc.

Greetings

Roger
0
Sébastien Riccio Replied
Hello Roger,

Thanks you for testing. I'm really worried at this point as 95% of our customers are using SMTP and we activated DKIM (along with DMARC) for every domains as it is now mandatory if you don't want to be rejected by the big ones...

As a side effect it seems that, at least microsoft, is rejecting a mail if DKIM fails, even if SPF is correct. Not sure about gmail...

I opened a ticket about it about 5 hours ago, but no feedback from ST about it yet.

I fear the worse for our weekend here... Friday ST closed, so if no build with a fix available today it will be a hell of a weekend to find a workaround in the meantime...
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Sébastien Riccio Replied
And not to mention that when we troubleshoot DKIM/SPF etc issues, we quickly create a test mailbox on their domain to send a mail to a verification service, and everything pass, so we tell the customer there is no apparent problem with it... :'(
Sébastien Riccio System & Network Admin https://swisscenter.com
1
Mxtoolbox reports a valid DKIM here....
1
Oliver Replied
I can also confirm the behavior with 8818.

If the user has set up access via the following protocols:
EAS, MAPI or EWS or via webmail
The recipient's message contains the following in the source code
smtp.mailfrom=senderdomain.tld; dkim=pass (signature has been verified)

If access via the following protocols is set up for the user:
POP3/SMTP or IMAP/SMTP
The message to the recipient contains the following in the source code
smtp.mailfrom=senderdomain.tld; dkim=fail (signature has not been verified)
0
Sébastien Riccio Replied
Thank you Oliver.

Brian are you on 8818 ?
Sébastien Riccio System & Network Admin https://swisscenter.com
1
Yes
1
Employee Replied
Employee Post
Hi guys,

Thanks for bringing this to our attention. We looked into this and found that deliveries from the SMTP clients include an authentication-results header. When the email is received by Microsoft, they are changing that header from authentication-results --> authentication-results-original. They're doing this change before verifying the DKIM signature, which is causing DKIM to fail for those recipients. 

To mitigate that handling, we've made a change in code to always exclude the authentication-results header from DKIM signing. This change will be included in our upcoming release of SmarterMail. 

In the meantime, an immediate workaround you can implement is to modify your DKIM settings to exclude that header, like so:
1
Sébastien Riccio Replied
Hello Andrea,

Thank you for the infos. I don't understand though what is related to Microsoft here.

I have DKIM failing when sending mail through SMTP to check-auth@verifier.port25.com or to mail-tester.com.
No Microsoft involved here AFAIK.

Kind regards
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Sébastien Riccio Replied
Hmm, however as we are using an outgoing gateway, the authentication-results is replaced by the outgoing gateway, which will indeed fail if it is taken into account for DKIM signing.

We had the headers set to "Non-repeatable headers". Will try what you are suggesting (which makes sense)

----------------------------------------------------------
DKIM check details:
----------------------------------------------------------
Result:         fail (signature doesn't verify)
ID(s) verified: 

Canonicalized Headers:
    mime-version:1.0'0D''0A'
    user-agent:eM_Client/9.2.2157.0'0D''0A'
    reply-to:"madjik@xxx.ch"'20'<madjik@xxx.ch>'0D''0A'
    message-id:<em8571c293-a318-4d97-81b1-d6043756c528@8042f965.com>'0D''0A'
    date:Thu,'20'29'20'Feb'20'2024'20'15:31:20'20'+0000'0D''0A'
    subject:test'20'testse'0D''0A'
    to:"check-auth@verifier.port25.com"'20'<check-auth@verifier.port25.com>'0D''0A'
    from:"madjik@xxx.ch"'20'<madjik@xxx.ch>'0D''0A'
    authentication-results:mta-gw.xxx.net;'20'auth=pass'20'(preauth)'0D''0A'
    dkim-signature:v=1;'20'a=rsa-sha256;'20'd=xxx.ch;'20's=8D9917C959C30CF;'20'c=relaxed/relaxed;'20't=1709220689;'20'h=mime-version:user-agent:reply-to:message-id:date:subject:to:from:'20'authentication-results;'20'bh=vCIFX/maFoyGBSBxIG6gNODd3oBZ3n4d2ywcqSUe7RU=;'20'b=

EDIT: It seems ok with the suggested settings (tested on one domain). We have ~5000 domains, how can we propagate the change to all of them ? Thanks.

Kind regards


Sébastien Riccio System & Network Admin https://swisscenter.com
1
Kyle Kerst Replied
Employee Post Marked As Answer
@Sébastien - you should be able to just move to the new release we just published. I did not need to manually intervene for any of the domains I was seeing problems on previously, and they now pass DKIM checks to 0365. Can you give this a try for us after backing up your environment please?
Kyle Kerst IT Coordinator SmarterTools Inc. www.smartertools.com
0
Sébastien Riccio Replied
Yes... Thanks for the update Kyle.
But now the server spool is dead. No outgoing mail or incoming mail are working....
EDIT: it seems it finally unlocked itself... after 7-8 minutes... phew

EDIT2: The DKIM issue with SMTP seems resolved. Thank you for this!
Sébastien Riccio System & Network Admin https://swisscenter.com
1


Tested on release 8818 on mailtester.com 
0
Sébastien Riccio Replied
Hello Brian,

You are lucky :) I guess it depends which headers you are using for DKIM signing and/or if you are using an outgoing gateway (which will alter the authentication-results header).

Kind regards
Sébastien Riccio System & Network Admin https://swisscenter.com
1
We are not using an outbound GW and I bet thats the culprit :)
0
Douglas Foster Replied
NOTE:   You can update the header fields on your DKIM configuration, but it does not take effect, so the workaround does not work.   Bug confirmed on my open ticket #359-2D5C47BF-0B0A.   In the interim, I suspect that the change will take effect if  you do a key rollover, but I have not tried this.

Authentication-Results fields are SUPPOSED to be removed at an organizational boundary, either by the sender or the recipient, so they should NEVER be included in a DKIM signature's header list.  (Obviously, renaming is an alternative to removal.)  This is in the RFC, and it is a security protection against deception..

ARC-Authentication-Results, by comparison, is intended to survive an organizational boundary, so it could be DKIM-signed.   However, the nature of the ARC chain means that DKIM signing of ARC headers will be redundant and therefore unnecessary.
0
Sébastien Riccio Replied
You are right Douglas.

After upgrading to latest build we still have some customers having issue with DKIM when they use SMTP. This particular user is using fastmail as their mail hub and configured their smartermail account as a secondary address on fastmail (using SMTP).

When they send mails from fastmail through their SM account via SMTP, the DKIM signing fails.

I saw that fastmail is adding quite some X-* headers and I suspect these are used for the signing but some get altered on the way.

So I changed this customer DKIM configuration to use only recommended fields as per the RFC: https://dkim.org/specs/rfc4871-dkimbase.html#rfc.section.5.5
But SM continues to use the other X-* fields for signing. I tried a domain reload... no changes.

However in the domain settings.json the config change is saved:


        "dkim_activation_forced": false,
        "dkim_active": true,
        "dkim_canonicalization_algorithm_body": "relaxed",
        "dkim_canonicalization_algorithm_header": "relaxed",
        "dkim_header_field_option": "allspecified",
        "dkim_header_fields": [
            "From",
            "Sender",
            "Reply-To",
            "Subject",
            "Date",
            "Message-ID",
            "To",
            "Cc",
            "MIME-Version",
            "Content-Type",
            "Content-Transfer-Encoding",
            "Content-ID",
            "Content-Description",
            "Resent-Date",
            "Resent-From",
            "Resent-Sender",
            "Resent-To",
            "Resent-Cc",
            "Resent-Message-ID",
            "In-Reply-To",
            "References",
            "List-Id",
            "List-Help",
            "List-Unsubscribe",
            "List-Subscribe",
            "List-Post",
            "List-Owner",
            "List-Archive"
        ],
I'm not very hot to try to change/rollover the key or anything that need we update their DNS record.

I guess a stop/start of SM could maybe do the trick, but also not hot to do a service interruption to reload dkim settings...

EDIT1: Well I went ahead this night and stopped / started the service in order to (try to) reload DKIM settings for a test domain. No luck either. When you change the selected headers for signing and save, it updates the domain's settings.json file correctly, but when you stop SmarterMail, it reverts it back to the previous settings.
Looks like when you change these settings it is reflected in the domain config file but not in "memory" and when you stop the service it dumps back the in-memory settings to the domain config file... :( 

EDIT2:
For the fastmail + SMTP + SM, It's not an header issue, the body is altered.
----------------------------------------------------------
DKIM check details:
----------------------------------------------------------
Result:         fail (wrong body hash: expected oXOqt5ZXsno8obedckABk7o6IHvt//pkwEj6+2hbKD8=)

If i try to send a "html" (without accented  chars in the body) mail through SMTP/SM from fastmail, the DKIM is FAIL.
If i try to send a "html" (with accented  chars in the body) mail through SMTP/SM from fastmail, the DKIM is PASS.
If I try to send a plain text through SMTP/SM from fastmail, the DKIM is PASS.

I did the same tests with another SMTP server (not SM) and all DKIM signing is PASS.

Looks like there are still issues with SMTP + DKIM signing on SmarterMail and looks like it is related to mail charset encoding.. :(
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Douglas Foster Replied
You still have too many fields in your DKIM configuration.   You never want to include fields that might be legimately added or altered in transit, so you should drop all of the RESENT-* fields, all of the LIST-* fields, and the Sender field.

Then upgrade to build 8825 to fix the problem with Authentication-Results being included in the header list.

Finally, try doing a key rollover to force SmarterMail to start using your revised header list.

These do not seem to be the immediate problem, but they will be the problem as soon as the others are resolved.
0
Sébastien Riccio Replied
Well these seems to be the recommended fields to take into consideration as per the RFC but, well, anyway the issue i'm facing is not related to the headers but the body hash being altered at some point. I suspect an encoding handling issue.

Historically we were using "All non-repeatable" fields until it started causing issue because of Authentication-Results.

Sébastien Riccio System & Network Admin https://swisscenter.com
0
Sébastien Riccio Replied
Btw, I tried to do a key rollover on a test domain, it didn't change anything for the headers. It still uses the old list of headers...

Sébastien Riccio System & Network Admin https://swisscenter.com
1
Douglas Foster Replied
I have also documented some problems with DKIM breakage caused by body modifications, and gave traced it to the delivery module.   One source of the problem was fixed in 8818, but some others are still being investigated.

Gmail is a good reference for optimal headers.  When I have compared header lists, their list is usually a superset of fields used by any other other sender. They oversign fields that should never appear twice, such as From, Date, To, and Subject.  But they do not sign everything.


0
Sébastien Riccio Replied
Hello Douglas, thanks for the information. Also nice to know they are still investigating some issue. I hope the one our customer is affected with will be fixed at same time.
Sébastien Riccio System & Network Admin https://swisscenter.com
2
Sébastien Riccio Replied
Updated to 8832

Some DKIM issues with body signing resolved, great.

But it still doesn't want to take into account the specified headers field I've set for signing. I tried with just From,To,Subject and it still takes all headers:

DKIM-Signature: v=1; a=rsa-sha256; d=cybermind.ch; s=vd8DC3C98C8FAA112;
c=relaxed/relaxed; t=1709878224;
h=from:date:subject:message-id:to:reply-to:mime-version;
Additionnaly now when sending mail from webmail I have this:
(my autocomplete list is gone and is not saving any new entry)


I am really getting TIRED of all this....
Sébastien Riccio System & Network Admin https://swisscenter.com
4
Brian Bjerring-Jensen Replied
Issues all the time.... every new update solves problems and introduces new ones or even old ones show up again.

Time wasted...
2
Sébastien Riccio Replied
Yes, unfortunately. The product has nice features and idea but all these issues ruins it :(
Sébastien Riccio System & Network Admin https://swisscenter.com
5
The problem is really that you cant trust an update to be better than the previous release.

And thats an issue...
5
J. LaDow Replied
This is why there needs to be a "STABLE / LTS Servicing Channel" -- as an "enterprise" product, administrators shouldn't be expected to be beta testers.  Exchange may be exponentially more expensive but the release stability says something...
MailEnable survivor / convert --
1
Keith Dovale Replied
To be quite honest I dont see why I need to pay 40k to get a feature of the product to work, by only upgrading, yes sure if I want new features certainly then I will pay, but to pay to get the original functionality to work is a load of bollocks. why must i pay 40k to get dkim working ? ridiculous

4
Douglas Foster Replied
You need support because you need your vendor to stay in business. If you have followed this blog for years, you should have some appreciation of the complexity of this undertaking.  POP, IMAP, MAPI, and EWS each have layers of complexity aggravated by time zone and language differences, and different client software products and product versions..  All this before you begin the issues with message exchange to other organizations, and figuring out what to trust and how to be trusted.  There is a reason that you have few alternatives to Microsoft 

Yes, I hate the instability in a critical product, but we dynoed Exchange knowing there were risks, but I have no regrets 

If you are running a mission critical mail system and not paying for support, you are a freeloader and I have no sympathy.
4
Matt Petty Replied
Employee Post
Keith, You haven't had a license in years. What are you on about? You're also on a 3 year old version complaining about a feature not working. Are you talking about DKIM?
Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com
0
Keith Dovale Replied
yup correct, I paid for a product that worked for many years, then an update breaks the working functionality, and then I need to pay again to get the FIX... I understand if I want new functionality and all the bells and whistles then sure keep an active support contract. I bought the licence to give mail functionality, ie pop imap, smtp mails sending and receiving, what goes with that is the functionality of dkim, and ssl to encrypt the connections. noting more or nothing less. if the basic product was working and all was going fine, I wouldnt need a new support contract, etc. 

Look at it in this perspective, if I bought a car, I would expect it to have a motor that works, breaks and steering that works, etc. But 4 years later an issue is picked up that the brakes are wearing out too quick, that the producer of the cars says, yes we picked up an issue, but you need to buy a new car to get the car with the fix so it wont happen.. Crazy, I would have assumed you would have a back patch for clients that have a not so older version of the software. And to be quite frank it was all working until it was patched and we only found out now that there were some issues once we instituted dmark rejects... google claims the mails are passing, but other mail servers are saying no the mails are failing..

TBO the cost of the 2 licences annually would take 2 months of full income from our company, so to relate that to trying to pay for other stuff like DC costs etc, we basically would be running at a loss and just close the company down. Maybe a few hunderd dollars to you is nothing but to us with an exchange rate of 20:1 it makes it a bit hard.

my one server which is 15x is working 100% no issues for years now, the 17x doesnt work. and to upgrade to get a fix is outta reach..
5
Oliver Replied
Wow, I'm really sick of hearing the comparison between cars and software.

I think an open source solution would be more suitable for your needs.

Reply to Thread