I'm using the latest SM, I thought I would take a look at the receiving msg from a yahoo account. I'm not seeing what your seeing. The domain portion of the email is being sent in my message. 

Maybe it's a setting in your server in SM.

I asked the ST support about that and was told there wasn't anything.  Will inquire again and note your comment.  Thank you!

Hello Vince, 

When you tested this sending to yahoo.com how did you send the message?

So far we are not seeing any issues when sending from webmail at all, 

Thank you

Hi Tony,

I'm not experiencing any problems.  I just responded to Mark's issue. 

He asked if anyone else was experiencing that problem so I thought i would try and see if it was happening to me.

It seems to be working fine for me, it is showing the full domain properly in the message-id when viewing the raw message from my yahoo account.

I sent using webmail to a yahoo account

FYI - I'm not sending through webmail, but through ASP.NET instead.

Hey Mark, 

Please open a ticket with us regarding this issue. We will then be able to look through your code and see if we see anything wrong. 

Kind Regards, 

Hey Mark, 

One other thing please let me know if this post helps. 

Kind Regards, 

Thanks for your reply, Zach.  I had an open ticket which recommended I post here.  I'll responsd to that existing ticket.

We are using an outgoing gateway to handle outgoing mails sents by our SmarterMail server. 

I checked the log of our gateway by curiosity, and we have thousands of:

Jul 25 01:07:04 mta-gw haraka[956]: [INFO] [-] [core] [outbound] Sending email as a transaction

Jul 25 01:07:04 mta-gw haraka[956]: [INFO] [-] [core] [outbound] Processing delivery for domain: xyzabc.com

That would make me think that when a mail is sent from a mail client,via SmarterMail's SMTP, if the message-id header is not generated by the mail client, SmarterMail won't add one.

Hopefully here our gateway adds one if it's missing so we don't have the message-id issue with Gmail.

It looks like Gmail recently reviewed their antispam checks and they are being more agressive on missing/non rfc compliant headers, also duplicate headers.

There is some discussions about it on the mailop mailing list:

For example it looks like Windows 10 mail app doesn't add the message-id header, so when relayed through SmarterMail they will likely missing a message-id.

The RFC though doesn't say that message-id is required but "SHOULD" be  added. So somewhere Gmail is not following the RFC on this point by forcing mail to contain a message-id

On the other side it wouldn't hurt to add it if it's missing. Most of the MTA seems to do this according to some ppl

Kind regards

 

Sébastien,

Thank you for that - very informative!  The issue does seem to be affecting a wider audience with Gmail policy changes (and possibly others/soon).  Hopefully responses like yours will drive the need for this to be addressed in a future SM update.

Thanks for your testing and findings on this everyone! Through further investigation it looks like Gmail is not using the standard RFC surrounding message-id behavior (not required per RFC, and this is causing those bounced deliveries as not all clients send these currently.) I was able to replicate these concerns using Windows Mail and a simple SMTP I have escalated a case to our development team to take a look at this and see if we can implement a workaround though that should keep Gmail and anyone else who requires these in the future happy. We'll keep you all posted on these issues as new information becomes available. 

Hello everyone, 

I wanted to let you know that Build 8250 has been released and addresses this issue. As a reminder, we recently found that Google is now requiring the message-id header for inbound delivery, and some email clients do not automatically append one to the message, since it is not required by RFC. However, to accommodate Google's change and help alleviate this issue, SmarterMail will now append a message-id header to outbound messages if one was not already included. Please upgrade to Build 8250 at your leisure, and let me know if this helps to alleviate the problem with deliveries to Gmail. 

SmarterMail Downloads

Kind regards,

What a wonderful surprise!  Will give this a try ASAP.  Thanks for ST's efforts :)

Good morning.  I upgraded to the 8250 build.  Unfortunately, I still see a potential issue.

What started this thread and my two tickets was that the "Message-ID" portion of the header wasn't missing, but was instead missing a complete domain name.  Here is the header I get now when using 8250:

        Message-ID: <80c66c72a91a40449a76bbc321706f0b@com>

So if your update is only checking for the presence of "Message-ID" and then creating one if it doesn't exist (which would be a step forward), then that doesn't address the issue I initially reported.  

The initial reported issue was that the domain name was incomplete within the "Message-ID" header.  Can you take another look and complete the domain name if it's an invalid domain?  It should use the domain through which the email was sent.

UNLESS - there is something I need to configure on my end with the new build.  The support response used the term "automatic" so I've not done anything except install it / restart mail service.

Please advise.

Hi Mark, 

Thank you for your feedback, and I apologize for not addressing this piece in my response yesterday. When the issue with Gmail rejecting messages was brought to development, they informed us that the message-id header does not require any specific formatting. It can be any GUID, as long as it does not contain dashes. 

SmarterMail already contained logic to apply a message-id on inbound emails. When an email comes into SmarterMail, and it doesn't contain a message-id, we apply a message-id in the format of "{GUID}@com". When looking into this issue with Gmail more, we were able to replicate an issue where Gmail was rejecting emails that didn't contain a message-id header at all. Therefore, the same logic for inbound emails was applied to outbound emails as well, using the same format of "{GUID}@com". 

In our testing, Gmail accepted these emails without complaint. (We sent many messages without a message-id and saw that Gmail intermittently blocked these messages with a complaint about the message-id. For the few messages that made it in, we saw that Gmail had added their own message-id. Then, on the same emails, we added a message-id in the format of "{GUID}@com" and did multiple deliveries again. All of those messages were successfully received in multiple Gmail accounts.)

Please let me know if you find that Gmail is still blocking messages that contain a message-id in the format of "{GUID}@com"? We have been unable to replicate this behavior and will need to look into this more if it occurs. 

Andrea, thanks for your prompt response to this.  If you look at my original ticket on this where Tony did some research, you'll see he seemed to indicate this incomplete domain in the "Message-ID" header portion was an/the issue.

There may not be further need to address this, but would ask you confer with him for an update if possible.

Hi Mark, 

We did initially believe that the {GUID}@com format was a malformed message-id. However, when discussing this with development, they assured us that it is not malformed. RFC guidelines state that the email client delivering the email is responsible for appending the message-id header. However, the header is not required, and there is not a specific format that's needed. 

At this time, I believe that our change (adding a message-id to outbound emails if it's not already included) should be enough to mitigate the issue with Gmail now requiring a message-id. However, if you find that Gmail is still rejecting messages due to the message-id FORMAT, please let us know so we can consider other options. 

Kind regards,

Ok, will do.  Thank you.

Andrea, I'm curious, if you're going to the trouble of adding a message ID in the first place, why wouldn't you go the whole distance and add the domain name instead of just @com?  

First of all, it just makes sense from a readability (and logical) standpoint.  When we first looked through the logs and saw @com, we assumed there was an error.  And in the case of any other TLD than .com, it's going to generate more questions for anyone who views headers.

Second, it gets ahead of the curve if or when the RFC is updated to add the domain name to the ID.  It seems that adding @com and just stopping there, was only going the minimum distance to get the task over with, and not looking further down the road.  Surely adding the correct domain name would be trivial, right?  I mean, you've come this far; why not go that extra tiny step?

 

Jay seems to make a good point.  Wonder about this and also if there is or could/should be any user options in the SM configuration?

Regarding...

But what if the message-id added by the email client is damaged, incomplete, or junk?

Example:  Message-ID: <105c8f319ef74f63a8d2202b1a55dcc7@com>

After updating to Build 8250 the problem still occurs. 

If the message-id is malformed I want SmarterMail to correct it.

Please correct the ".com" to "@mydomain.com".

The Gmail rejections are hurting our clients.

Hello,

I think if the message-id generated by the submitter of the mail is invalid, it should not be changed by the MTA (SmarterMail in this case).

Because the application sending the mail would not be able to track any bounces or answers to that particular mail.

It's up to the client application to generate a "valid" message id if it wants to add one. If it doesn't add one that means that it doesn't need it for it's internal stuff so adding one won't hurt, but modifying it could cause more trouble that it fixes.

Also if gmail is considering a message-id invalid if it doesn't match somestring@domain.tld they are somehow wrong about it, because AFAIK there is no obligation definied in RFC for this format.

Is there an option in the library/software you use to send the mail, to define the message-id format ?

Kind regards

I agree with Sebastien

The gmail.com admin appears to have removed the Message ID filter/block sometime last week.

The messages generated via DIMAC Jmail in our Classic ASP application and sent via SmarterMail are not being rejected at gmail.com.

From: "info@generalmeetings.net" <info@generalmeetings.net>

To: "douglasbrantley@gmail.com" <douglasbrantley@gmail.com>

Subject: AFP-SBV - Event Registration - Pending Payment

Date: Sat, 3 Sep 2022 10:23:52 -0700

Reply-To: info@generalmeetings.net

MIME-Version: 1.0

Content-Type: text/html; charset="utf-8"

Content-Transfer-Encoding: quoted-printable

Return-Path: <info@generalmeetings.net>

Sender: "info@generalmeetings.net" <info@generalmeetings.net>

Appreciate the update.