3
Build 8797 Strange Characters in HTML formatted email
Problem reported by Ron Raley - 2/11/2024 at 4:02 PM
Submitted
After upgrading to Build 8797.  This particular example occurs in Gmail app.


=09 and =20 are just a few. There are more.

33 Replies

Reply to Thread
0
Ron Raley Replied
Our HTML code may be old. Is SmarterMail demanding a new charset?
1
Brian Bjerring-Jensen Replied
Have you tried unicode UTF8??
0
Ron Raley Replied
Yes, UTF-8 is running in each email header

Subject: Apparatus Status: New Entry
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Mailer: ColdFusion 2018 Application Server
0
Ron Raley Replied
We have discover that this issue only occurs when FORWARDING from a SmarterMail e-mail address to a private email address (Gmail).
1
Ron Raley Replied
We have started a support request.  Will advise.
4
Ron Raley Replied
Confirmed bug.
0
mgreider Replied
FYI - We are seeing a similar issue on our side too.
0
Sébastien Riccio Replied
Hello,

multiple customers also reported the same issue. We're on build 8832 ...


Sébastien Riccio System & Network Admin https://swisscenter.com
0
Ron Raley Replied
- Build 8818 (Feb 22, 2024)
- Added: Authenticated Received Chain (ARC) Verification and Signing

This was "supposed" to fix the issue, per SmarterTools.

We were able to fix the issue by sending our alerts with "Content-Transfer-Encoding" value "base64"
0
SmP Replied
We're seeing this issue on build 8832. We really can't send everything with base64.
0
SmP Replied
We observed this when emailing signing is enabled; when it's disabled the issue goes away.
0
Sébastien Riccio Replied
We really need a definitive fix for this ASAP, we have dozens of customer complaints...
Sébastien Riccio System & Network Admin https://swisscenter.com
1
mgreider Replied
We have an open ticket on this now, and here is what we've uncovered. Not sure if it will help you all or not. Still waiting on the full resolution from the SM team, but we have a workaround in place.

We were on 8797 and migrated to 8832. At this point, any HTML email that we sent from SmarterMail to Gmail, O365 and some other random email servers was mangled.

When we set up archiving and looked at the eml from SmarterMail, everything looked normal. On another SmarterMail server, when receiving the email, everything also looked normal. When looking at the resulting .eml at Google or O365, we noticed these headers:

Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

But then in the body of the email, it was escaping = with =3D and the like.

What we did, was we changed our outbound emails to have mixed parts, a plain text and html version. The plain text was just a simple one liner that said "Your client does not support HTML emails" or something like that.

Now, in Gmail and O365, they received it as a multipart email. The plain text had headers:

Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

And the html had headers:

Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

And the usual escaping of the body. This made everything render like normal.

TLDR: We changed our HTML only emails to be multipart, leaving the same HTML but including a plain text part and that has done the trick temporarily.
0
mgreider Replied
Also Ron - Ours is also coming from ColdFusion. 
0
Sébastien Riccio Replied
SmP, when you say:

We observed this when emailing signing is enabled; when it's disabled the issue goes away.
You mean DKIM signing ? So enabling DKIM changes the body content and/or the encoding of a mail ? :/
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Ron Raley Replied
mgreider, add this inside your <cfmail tag>

<cfmailparam name="Content-Transfer-Encoding" value="base64">
0
mgreider Replied
I did try to base64 encode the contents and then set that header. Can you set that header even if it's just regular text or plain/html?
0
SmP Replied
When DKIM signing is turned off, this behavior goes away; when DKIM is turned on HTML messages sent via SMTP on SmarterMail by third party services (such as a WordPress website) get a bunch of encoding/line break issues.
0
mgreider Replied
FYI - The SM support team has not been able to duplicate my issue. Anyone else have additional use cases?
0
Kyle Kerst Replied
Employee Post
I was just able to replicate this but only in a very specific setup:

- 7 bit transfer encoding used in the SMTP session
- DKIM enabled on the sending domain
- Sending to an account that uses Automated Forwarding to relay the message on to outside recipients

With this setup I was able to replicate encoding behaviors and the table issues seen on the ticket I have open on this, so I'm going to get a task escalated to development now!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
SmP Replied
Very curious as we can replicate even without forwarding (it occurs even when sent to an external recipient without forwarding).
0
Kyle Kerst Replied
Employee Post
Agreed! Unfortunately I can't replicate under the conditions noted above or in the ticket I have open, but I think the replication I have here is close enough that development should have something to look at. In the meantime, I did find changing the transfer encoding to base64 resolves the issue as well.
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
2
Sébastien Riccio Replied
Hello,

I can reproduce a related issue with a simple test with PHPMailer:

<?php
require_once "vendor/autoload.php";

$mail = new \PHPMailer\PHPMailer\PHPMailer();
$mail->SMTPDebug = 3;
$mail->isSMTP();
$mail->Host = "smartermail.server.com";
$mail->SMTPAuth = true;
$mail->Username = "user@domain.tld";
$mail->Password = "uglypassword";
$mail->SMTPSecure = "tls";
$mail->Port = 587;
$mail->From = "user@domain.tld";
$mail->FromName = "John Doe";
$mail->addAddress("some@recipient.tld");
$mail->CharSet('utf-8');
$mail->isHTML(true);
$mail->Subject = "Test HTML mail";
$mail->Body = "<i>This is a test éoéoéo!</i>";

if (!$mail->send()) {
    echo "Mailer Error: " . $mail->ErrorInfo;
} else {
    echo "Message has been sent successfully";
}

Transaction log:

2024-03-13 07:35:06 Connection: opening to mail01.swisscenter.com:587, timeout=300, options=array()
2024-03-13 07:35:06 Connection: opened
2024-03-13 07:35:06 SERVER -> CLIENT: 220 mail01.swisscenter.com SwissCenter Mail Server; Wed, 13 Mar 2024 08:35:06 +01:00; Your IP: 94.103.96.188
2024-03-13 07:35:06 CLIENT -> SERVER: EHLO cybermind.ch
2024-03-13 07:35:06 SERVER -> CLIENT: 250-mail01.swisscenter.com Hello [94.103.96.188]250-SIZE 69905066250-AUTH LOGIN CRAM-MD5250-STARTTLS250-8BITMIME250-DSN250 OK
2024-03-13 07:35:06 CLIENT -> SERVER: STARTTLS
2024-03-13 07:35:06 SERVER -> CLIENT: 220 Start TLS negotiation
2024-03-13 07:35:06 CLIENT -> SERVER: EHLO cybermind.ch
2024-03-13 07:35:06 SERVER -> CLIENT: 250-mail01.swisscenter.com Hello [94.103.96.188]250-SIZE 69905066250-AUTH LOGIN CRAM-MD5250-8BITMIME250-DSN250 OK
2024-03-13 07:35:06 CLIENT -> SERVER: AUTH CRAM-MD5
2024-03-13 07:35:06 SERVER -> CLIENT: 334 PC0xMDY4MjgxMjUyLjYzODQ1OTE1NzA2OTEzOTAwMEBtYWlsMDEuc3dpc3NjZW50ZXIuY29tPg==
2024-03-13 07:35:06 CLIENT -> SERVER: [credentials hidden]
2024-03-13 07:35:06 SERVER -> CLIENT: 235 Authentication successful
2024-03-13 07:35:06 CLIENT -> SERVER: MAIL FROM:<madjik@cybermind.ch>
2024-03-13 07:35:06 SERVER -> CLIENT: 250 OK <madjik@cybermind.ch> Sender ok
2024-03-13 07:35:06 CLIENT -> SERVER: RCPT TO:<sr@madjix.ch>
2024-03-13 07:35:06 SERVER -> CLIENT: 250 OK <sr@madjix.ch> Recipient ok
2024-03-13 07:35:06 CLIENT -> SERVER: DATA
2024-03-13 07:35:06 SERVER -> CLIENT: 354 Start mail input; end with <CRLF>.<CRLF>
2024-03-13 07:35:06 CLIENT -> SERVER: Date: Wed, 13 Mar 2024 08:35:06 +0100
2024-03-13 07:35:06 CLIENT -> SERVER: To: sr@madjix.ch
2024-03-13 07:35:06 CLIENT -> SERVER: From: Cybermind Corp <madjik@cybermind.ch>
2024-03-13 07:35:06 CLIENT -> SERVER: Subject: Test HTML mail
2024-03-13 07:35:06 CLIENT -> SERVER: Message-ID: <XiTDOQeIP3ZMYXlMN5GlQtEDuxEECWUmpJQrOms@cybermind.ch>
2024-03-13 07:35:06 CLIENT -> SERVER: X-Mailer: PHPMailer 6.9.1 (https://github.com/PHPMailer/PHPMailer)
2024-03-13 07:35:06 CLIENT -> SERVER: MIME-Version: 1.0
2024-03-13 07:35:06 CLIENT -> SERVER: Content-Type: text/html; charset=iso-8859-1
2024-03-13 07:35:06 CLIENT -> SERVER: Content-Transfer-Encoding: 8bit
2024-03-13 07:35:06 CLIENT -> SERVER:
2024-03-13 07:35:06 CLIENT -> SERVER: <i>This is a test éoéoéo!</i>
2024-03-13 07:35:06 CLIENT -> SERVER:
2024-03-13 07:35:06 CLIENT -> SERVER: .
2024-03-13 07:35:06 SERVER -> CLIENT: 250 OK
2024-03-13 07:35:06 CLIENT -> SERVER: QUIT
2024-03-13 07:35:06 SERVER -> CLIENT: 221 Service closing transmission channel
2024-03-13 07:35:06 Connection: closed
Message has been sent successfully
Received result when DKIM is enabled on sending domain:


When DKIM is disabled:


Received result when adding $mail->Encoding = 'quoted-printable' or 'base64' (both with DKIM off or on):


So seems DKIM enabling is affecting the issue, also it seems that it is now needed to specify a different encoding to get  correct results. This is a regression from previous builds as this was not needed.

We can't really tell all our clients to suddenly modify their applications to change the content encoding, when it was perfectly working before...
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Chris Danks Replied
we upgraded from build 8825 (29th Feb 2024) to build 8832 (7th March 2024) and now we have our clients reporting random characters are in their outbound emails.
0
Kyle Kerst Replied
Employee Post
Thanks for your follow up on this Sébastien, I'm going to incorporate your test method into our open task on this so they have more than one avenue to reproduce the issue as needed. This task is to investigate why the change in encoding is required, so I don't anticipate this being a long term requirement. 
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
Kyle Kerst Replied
Employee Post
I was able to QC fixes provided by development for these encoding concerns and verified (via my own scripts and those provided by the community) that this issue has been resolved. If you need this fix sooner than our next/upcoming release please create a ticket and we'll get that over to you right away. Thanks for your help on this everyone!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
1
SmP Replied
Kyle, any word on when that next release will hit?
0
Sébastien Riccio Replied
Kyle, we have a ticket open for this issue but did not received the fix. Can you please do something about it ?
Sébastien Riccio System & Network Admin https://swisscenter.com
0
Kyle Kerst Replied
Employee Post
@SmP: I am not positive on when the release is scheduled but we generally shoot for Thursdays. 
@Sébastien: I'd be happy to!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
SmP Replied
Refreshing the download page like it's my job!
0
Sébastien Riccio Replied
Hello,

While the latest build seems to hopefully fix this issue, I have some question though.

When doing the tests, I noticed that SmarterMail converts all mails submitted through SMTP to "Content-Transfer-Encoding: base64"

For example when submitting a test with PHPMailer where it's originally "Content-Transfer-Encoding: 8bit" 

2024-03-18 01:40:26 CLIENT -> SERVER: X-Mailer: PHPMailer 6.9.1 (https://github.com/PHPMailer/PHPMailer)
2024-03-18 01:40:26 CLIENT -> SERVER: MIME-Version: 1.0
2024-03-18 01:40:26 CLIENT -> SERVER: Content-Type: text/html; charset=utf-8
2024-03-18 01:40:26 CLIENT -> SERVER: Content-Transfer-Encoding: 8bit
The received mail shows this.

X-Mailer: PHPMailer 6.9.1 (https://github.com/PHPMailer/PHPMailer)
MIME-Version: 1.0
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64
So it looks SM is re-encoding the content on the fly. While it doesn't seems to pose any problem that I know yet, I am wondering why SM is doing this.

I tested with different other SMTP relays and the Content-Transfer-Encoding is unaltered.

X-Mailer: PHPMailer 6.9.1 (https://github.com/PHPMailer/PHPMailer)
MIME-Version: 1.0
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
Any explanation about why SM is converting the content would be appreciated.

Kind regards

Sébastien Riccio System & Network Admin https://swisscenter.com
0
Kyle Kerst Replied
Employee Post
Great question Sébastien, I'll check in with development and let you know what I find out. Glad to hear its resolved though!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
Kyle Kerst Replied
Employee Post
This conversion is intentional Sébastien. Not all email servers can accept content that includes 8-bit encoded content, so SmarterMail converts the encoding to better support a range of receiving email servers. For messages that have non-ASCII characters in sections that are originally encoded with 8-bit encoding, this means that the section will either be converted to quoted-printable or base64.
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com

Reply to Thread