4
message-id (Gmail rejecting email)
Idea shared by MARK - 7/14/2022 at 2:19 PM
Completed
Wonder if others are experiencing this... in the recent week, Gmail (Yahoo and some others) have begun rejecting our email.  The ST support points to the "message-id" as the culprit.  Our emails lack the full domain name sending the email and we wind up with a header message-id something like:

  • message-id: <c04cb33b18584c2dadad1381d559c973@com>
Where it should read something more like:
  • message-id: <c04cb33b18584c2dadad1381d559c973@mydomain.com>

Not sure exactly what controls the generation of this header piece, whether that's SM, IIS, registrar, or receiving email client.

I've tried sending from my server using two technologies, one of which is ASP.NET; the other is a CGI program.

Both wind up going through the only "email server" software, SM however.  The receiver headers both resemble the partial domain in the message-id.

Would welcome any feedback others might be experiencing and/or where resolution may lie.

23 Replies

Reply to Thread
0
I'm using the latest SM, I thought I would take a look at the receiving msg from a yahoo account. I'm not seeing what your seeing. The domain portion of the email is being sent in my message. 
Maybe it's a setting in your server in SM.
0
I asked the ST support about that and was told there wasn't anything.  Will inquire again and note your comment.  Thank you!
0
Tony Scholz Replied
Employee Post
Hello Vince, 

When you tested this sending to yahoo.com how did you send the message?

So far we are not seeing any issues when sending from webmail at all, 

Thank you
Tony Scholz
Technical Support Specialist
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Hi Tony,

I'm not experiencing any problems.  I just responded to Mark's issue. 
He asked if anyone else was experiencing that problem so I thought i would try and see if it was happening to me.

It seems to be working fine for me, it is showing the full domain properly in the message-id when viewing the raw message from my yahoo account.
I sent using webmail to a yahoo account


0
FYI - I'm not sending through webmail, but through ASP.NET instead.
0
Zach Sylvester Replied
Employee Post
Hey Mark, 

Please open a ticket with us regarding this issue. We will then be able to look through your code and see if we see anything wrong. 

Kind Regards, 
Zach Sylvester
Technical Support Specialist
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Zach Sylvester Replied
Employee Post
Hey Mark, 

One other thing please let me know if this post helps. 


Kind Regards, 
Zach Sylvester
Technical Support Specialist
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Thanks for your reply, Zach.  I had an open ticket which recommended I post here.  I'll responsd to that existing ticket.
1
We are using an outgoing gateway to handle outgoing mails sents by our SmarterMail server. 
I checked the log of our gateway by curiosity, and we have thousands of:

Jul 25 01:07:04 mta-gw haraka[956]: [INFO] [-] [core] [outbound] Sending email as a transaction
Jul 25 01:07:04 mta-gw haraka[956]: [INFO] [-] [core] [outbound] Adding missing Message-Id header
Jul 25 01:07:04 mta-gw haraka[956]: [INFO] [-] [core] [outbound] Processing delivery for domain: xyzabc.com

That would make me think that when a mail is sent from a mail client,via SmarterMail's SMTP, if the message-id header is not generated by the mail client, SmarterMail won't add one.

Hopefully here our gateway adds one if it's missing so we don't have the message-id issue with Gmail.

It looks like Gmail recently reviewed their antispam checks and they are being more agressive on missing/non rfc compliant headers, also duplicate headers.

There is some discussions about it on the mailop mailing list:

For example it looks like Windows 10 mail app doesn't add the message-id header, so when relayed through SmarterMail they will likely missing a message-id.

The RFC though doesn't say that message-id is required but "SHOULD" be  added. So somewhere Gmail is not following the RFC on this point by forcing mail to contain a message-id

On the other side it wouldn't hurt to add it if it's missing. Most of the MTA seems to do this according to some ppl

Kind regards
 

Sébastien Riccio
System & Network Admin

2
Sébastien,

Thank you for that - very informative!  The issue does seem to be affecting a wider audience with Gmail policy changes (and possibly others/soon).  Hopefully responses like yours will drive the need for this to be addressed in a future SM update.
4
Kyle Kerst Replied
Employee Post
Thanks for your testing and findings on this everyone! Through further investigation it looks like Gmail is not using the standard RFC surrounding message-id behavior (not required per RFC, and this is causing those bounced deliveries as not all clients send these currently.) I was able to replicate these concerns using Windows Mail and a simple SMTP I have escalated a case to our development team to take a look at this and see if we can implement a workaround though that should keep Gmail and anyone else who requires these in the future happy. We'll keep you all posted on these issues as new information becomes available. 
Kyle Kerst
Technical Support Specialist
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
1
Andrea Free Replied
Employee Post
Hello everyone, 

I wanted to let you know that Build 8250 has been released and addresses this issue. As a reminder, we recently found that Google is now requiring the message-id header for inbound delivery, and some email clients do not automatically append one to the message, since it is not required by RFC. However, to accommodate Google's change and help alleviate this issue, SmarterMail will now append a message-id header to outbound messages if one was not already included. Please upgrade to Build 8250 at your leisure, and let me know if this helps to alleviate the problem with deliveries to Gmail. 

SmarterMail Downloads

Kind regards,

Andrea Free
SmarterTools Inc.
877-357-6278

www.smartertools.com

0
What a wonderful surprise!  Will give this a try ASAP.  Thanks for ST's efforts :)
2
Good morning.  I upgraded to the 8250 build.  Unfortunately, I still see a potential issue.

What started this thread and my two tickets was that the "Message-ID" portion of the header wasn't missing, but was instead missing a complete domain name.  Here is the header I get now when using 8250:

        Message-ID: <80c66c72a91a40449a76bbc321706f0b@com>

So if your update is only checking for the presence of "Message-ID" and then creating one if it doesn't exist (which would be a step forward), then that doesn't address the issue I initially reported.  

The initial reported issue was that the domain name was incomplete within the "Message-ID" header.  Can you take another look and complete the domain name if it's an invalid domain?  It should use the domain through which the email was sent.

UNLESS - there is something I need to configure on my end with the new build.  The support response used the term "automatic" so I've not done anything except install it / restart mail service.

Please advise.
1
Andrea Free Replied
Employee Post
Hi Mark, 

Thank you for your feedback, and I apologize for not addressing this piece in my response yesterday. When the issue with Gmail rejecting messages was brought to development, they informed us that the message-id header does not require any specific formatting. It can be any GUID, as long as it does not contain dashes. 

SmarterMail already contained logic to apply a message-id on inbound emails. When an email comes into SmarterMail, and it doesn't contain a message-id, we apply a message-id in the format of "{GUID}@com". When looking into this issue with Gmail more, we were able to replicate an issue where Gmail was rejecting emails that didn't contain a message-id header at all. Therefore, the same logic for inbound emails was applied to outbound emails as well, using the same format of "{GUID}@com". 

In our testing, Gmail accepted these emails without complaint. (We sent many messages without a message-id and saw that Gmail intermittently blocked these messages with a complaint about the message-id. For the few messages that made it in, we saw that Gmail had added their own message-id. Then, on the same emails, we added a message-id in the format of "{GUID}@com" and did multiple deliveries again. All of those messages were successfully received in multiple Gmail accounts.)

Please let me know if you find that Gmail is still blocking messages that contain a message-id in the format of "{GUID}@com"? We have been unable to replicate this behavior and will need to look into this more if it occurs. 

Andrea Free
SmarterTools Inc.
877-357-6278

www.smartertools.com

0
Andrea, thanks for your prompt response to this.  If you look at my original ticket on this where Tony did some research, you'll see he seemed to indicate this incomplete domain in the "Message-ID" header portion was an/the issue.

There may not be further need to address this, but would ask you confer with him for an update if possible.
1
Andrea Free Replied
Employee Post
Hi Mark, 

We did initially believe that the {GUID}@com format was a malformed message-id. However, when discussing this with development, they assured us that it is not malformed. RFC guidelines state that the email client delivering the email is responsible for appending the message-id header. However, the header is not required, and there is not a specific format that's needed. 

At this time, I believe that our change (adding a message-id to outbound emails if it's not already included) should be enough to mitigate the issue with Gmail now requiring a message-id. However, if you find that Gmail is still rejecting messages due to the message-id FORMAT, please let us know so we can consider other options. 

Kind regards,

Andrea Free
SmarterTools Inc.
877-357-6278

www.smartertools.com

0
Ok, will do.  Thank you.
2
Andrea, I'm curious, if you're going to the trouble of adding a message ID in the first place, why wouldn't you go the whole distance and add the domain name instead of just @com?  

First of all, it just makes sense from a readability (and logical) standpoint.  When we first looked through the logs and saw @com, we assumed there was an error.  And in the case of any other TLD than .com, it's going to generate more questions for anyone who views headers.

Second, it gets ahead of the curve if or when the RFC is updated to add the domain name to the ID.  It seems that adding @com and just stopping there, was only going the minimum distance to get the task over with, and not looking further down the road.  Surely adding the correct domain name would be trivial, right?  I mean, you've come this far; why not go that extra tiny step?
 
1
Jay seems to make a good point.  Wonder about this and also if there is or could/should be any user options in the SM configuration?
0
Regarding...

Build 8250 (Aug 3, 2022)

  • Changed: To accommodate mail servers that require a message-id header in order to accept inbound delivery (e.g. Gmail), SmarterMail will now append a message-id header on outbound messages, if one was not added by the email client.
But what if the message-id added by the email client is damaged, incomplete, or junk?

Example:  Message-ID: <105c8f319ef74f63a8d2202b1a55dcc7@com>

After updating to Build 8250 the problem still occurs. 

If the message-id is malformed I want SmarterMail to correct it.
Please correct the ".com" to "@mydomain.com".

The Gmail rejections are hurting our clients.
4
Hello,

I think if the message-id generated by the submitter of the mail is invalid, it should not be changed by the MTA (SmarterMail in this case).

Because the application sending the mail would not be able to track any bounces or answers to that particular mail.

It's up to the client application to generate a "valid" message id if it wants to add one. If it doesn't add one that means that it doesn't need it for it's internal stuff so adding one won't hurt, but modifying it could cause more trouble that it fixes.

Also if gmail is considering a message-id invalid if it doesn't match somestring@domain.tld they are somehow wrong about it, because AFAIK there is no obligation definied in RFC for this format.

Is there an option in the library/software you use to send the mail, to define the message-id format ?

Kind regards
Sébastien Riccio
System & Network Admin

0
I agree with Sebastien

Reply to Thread