Outgoing gateway: The handshake failed due to an unexpected packet format
Question asked by Thu Nguyen - 8/6/2020 at 11:47 AM
This is my main mail server: 
And this is outgoing gateway mail server:

In's smartermail gateway config:
Server's address:
Port: 80
Encryption: SSL
Status: Enabled
Type: round robin
Enabled authentication:
Username: Administrator
Password: xxxxxxx
Smartermail gateway:
Enabled smartermail gateway mode
Smartermail URL: 
Smartermail Username: Administrator 
Smartermail Password: xxxxxxx 

Then I create an email in to send out, delivery.log show me this:

11:29:52.102 [94002] Connection to from succeeded (Id: 1)
11:29:52.102 [94002] Exception: The handshake failed due to an unexpected packet format.
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
   at MailService.RelayServer.Clients.SMTP.ClientConnectionSync.InitiateSsl(Boolean validateAllCerts)
   at MailService.RelayServer.Clients.SMTP.SmtpClientSession.Initialize(String senderAddress, List`1 recipientList, String destServerIP)
11:29:52.102 [94002] Failed to send to IP , error string: Attempted to send the message to the following IPs: 
11:29:52.117 [94002] Removed from RemoteDeliveryQueue (0 queued or processing)

Can you help me on this error?

3 Replies

Reply to Thread
Sébastien Riccio Replied
Marked As Answer
Just my analysis on this.
It's look you're using SmarterMail for the outgoing gateway and have enabled the SmarterMail gateway mode.

According to the log:
11:29:52.102 [94002] Connection to from succeeded (Id: 1) 

It tries to connect on port 80, but I guess it should connect on port 25, unless you're running smtp on port 80 on the mail gateway.
Also I  don't think you should specify a username and password for authentication, only in the SmarterMail gateway.

To summarize:
Try with port 25 without authentication (except authentication for SmarterMail mode, if you're using SmarterMail as a gateway).

ps: It's only suppositions as we're not using SmarterMail, but other software for the outgoing gateway.

Sébastien Riccio
System & Network Admin

Thu Nguyen Replied
OK, thanks for your value advise. Look like it worked.
I changed from 80 to 25, encryption=none, and disable authentication mode.

Sébastien Riccio Replied
Cool. Glad I was able to help.
Sébastien Riccio
System & Network Admin

Reply to Thread