This message is no longer on the server. This message is a replacement for the missing message.

Question asked by Scarab - 6/10/2020 at 4:29 PM

Answered

Oh boy! Just upgraded to Build 7549 Enterprise version from Build 7242

Customers are contacting us non-stop because the majority of messages they are retrieving via POP or IMAP say as follows:

From: System Administrator
To:
Date: Wed, 10 Jun 2020 11:42:52 -0700
Subject: Message removed

This message is no longer on the server.

This message is a replacement for the missing message.

There is a X-ESETId: Header on the last line of these messages but we don't use ESET and neither do the customers affected.

These replacement messages are not displayed in the webmail interface, only in email clients using POP or IMAP.

Anyone know what this is, what is causing it, and how to stop it? There is no system message configured in SmarterMail that does this.

19 Replies

Reply to Thread

Scarab Replied

6/10/2020 at 5:09 PM

Found out that this is something from the Beta.

https://portal.smartertools.com/community/a93159/beta-pop3-message-removed.aspx

Apparently it crept back into the Final Release Build.

Sébastien Riccio Replied

6/10/2020 at 9:03 PM

Hello Scarab,

Do you have anything in IMAP and/or POP logs that looks like: "message could not be found" ?

In POP3 logs we have tons of:

RETR failed to FeedMessage for message 4 (UID: 15540) ERR: The group file F:\SmarterMail\Domains\xxx.ch\Users\gilles\Mail\Inbox\2020_5_27.grp is corrupt or a message could not be found in the header..

In IMAP:

[2020.06.11] 00:58:29.347 [188.x.255.x][37227440] Failed to fetch data item 17047 from GRP file for sabxxxxnr@saxxxna.ch in Inbox.

We're having customers reporting the same "This message is no longer on the server. " and our imap/pop3 logs are full of this.

Kind regards,

Sébastien

Sébastien Riccio System & Network Admin https://swisscenter.com

Employee Replied

6/11/2020 at 3:37 PM

Employee Post

Quick update here, I have opened up a ticket with Scarab, and it looks like Sebastien is already working with us on his issue. I will update here when I have more information.

Sébastien Riccio Replied

6/12/2020 at 12:35 AM

Well, I was told that this come from an issue in earlier beta versions that affected the users mailbox but it can't be, because we updated our prod server straight from 7242 to the mapi release, no beta was insalled on our prod server!

Still no solution provided and we're in deep trouble... :(

Sébastien Riccio System & Network Admin https://swisscenter.com

Sébastien Riccio Replied

6/12/2020 at 12:49 AM

Multiple customers already moved to another service provider since we applied the update as we can't satisfy them with a stable service anymore. We must in emergency move some customers to another mail system so we don't lose them, it's really painful.

Sébastien Riccio System & Network Admin https://swisscenter.com

Scarab Replied

6/12/2020 at 9:52 AM

Josh, I responded to the Support Ticket with compressed archives of our Logs and multiple accounts. We did not install any Beta Builds in between versions.

Sebastien, you are correct. For POP users who are receiving a replacement message have the following in the detailed POP Logs "RETR failed to FeedMessage for message X (UID: XXXXX) ERR: The group file D:\SmarterMail\Domains\domain.tld\Users\username\Mail\Inbox\2020_6_10.grp is corrupt or a message could not be found in the header."

For IMAP users, especially MS Outlook users, they are getting the following error in their clients: "The UID of a message changed unexpectedly. This typically indicates a server bug. Your program may not function properly after this." In the detailed IMAP logs we have "Failed to fetch data item 21013 from GRP file for username@domain.tld in Inbox."

Tim Uzzanti Replied

6/12/2020 at 10:44 AM

Employee Post

We believe the issues is resolved in our upcoming release today. Its was a timing issue.

Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com

Scarab Replied

6/12/2020 at 11:36 AM

Thank Tim, we'll cross our fingers and give the updated version a whirl in the early morning hours of the morrow.

Sébastien Riccio Replied

6/13/2020 at 4:35 AM

Hello,

After updating to the latest release we're still seeing logs entries for corrupt grp in POP and IMAP logs.

We really need to find a solution, many customers are complaining and asking what is going on.

Can you please elaborate what do you mean with "a timing issue". We also need a confirmation that there is no possibility of lost e-mails because of this issue.

Multiple customers claimed to have lost content and asked us to restore their mailboxes from backups.

Kind regards.

Sébastien Riccio System & Network Admin https://swisscenter.com

Sébastien Riccio Replied

6/15/2020 at 2:24 AM

Hello, any news on this issue? We already got this morning a whole bunch of customers (most on POP3) having again new mails in their mail client with the content:

This message is no longer on the server.

This message is a replacement for the missing message.

It's really a mess and we have no idea what to do.

Scarab may I ask you if on your site the issue still persists too after the update to friday's build?

ST Please help us!

Sébastien Riccio System & Network Admin https://swisscenter.com

Scarab Replied

6/16/2020 at 4:08 PM

Sebastien,

Yes, we are still having the issue even after installing Build 7468.

ST opened a Trouble Ticket for us and we are working with them, providing them logs, accounts, and remote access so that they can better isolate the issue.

It does look like a transient issue. The accounts affected one day are not the same accounts affected the next day, and thankfully it's a small number of accounts affected (the most we've had are 11 POP accounts and 9 IMAP accounts, the low has been just 3 POP and 1 IMAP).

Sébastien Riccio Replied

6/16/2020 at 4:32 PM

Hi Scarab and thank you for your feedback. Based on our logs we have ~350 accounts affected on 20k total.

Most complaints come from POP users, but my guess is that this is not handled the same for IMAP and POP....

I have good hopes that ST team will find and fix the issue.

Kind regards

Sébastien Riccio System & Network Admin https://swisscenter.com

Sébastien Riccio Replied

6/16/2020 at 7:40 PM

Hmm Scarab, what's your server timezone set to, if I may ask ? Maybe I'm wrong but... I might have an idea about this.

Sébastien Riccio System & Network Admin https://swisscenter.com

Tim Uzzanti Replied

6/16/2020 at 8:53 PM

Employee Post

We have a hot fix on a couple customers servers. It is what I had mentioned previously, a timing issue which could cause an exception. In these situations the backend thinks the message count is one higher than it really is. The previous public release had part of the fix but it required an additional code change. Next release this will be fix will be included.

Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com

David Moller Replied

2/10/2022 at 11:15 AM

This is happening to most of my customers ever time I reboot the server. I am fully updated on the software, and would love to stop this from occurring as I need to do maintenance over the next few days on the server which will take a few reboots. Getting 100 calls, emails and texts from people telling me they are missing emails.

Tim Uzzanti Replied

2/10/2022 at 11:54 AM

Employee Post

I see you have no tickets since 2013? Might want to contact our support department. That seems like a server, storage, or hardware issue affecting SmarterMail.

Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com

Jay Dubb Replied

2/16/2022 at 9:24 PM

We've been seeing the same thing for the first time, and it started with 8055. Reboots generate dozens or hundreds of message-removed emails, and our phones start ringing. Our lead opened a support ticket, and was told to manually stop the SM service and wait for MailService.exe to disappear from Task Manager before rebooting.

That's ok for daytime, but we schedule automated reboots for the 3:00 a.m. hour when our clients are sleeping, and the nighttime on-call person does not have the level of access needed to stop services. (he has to escalate to an admin)

Kyle Kerst Replied

2/17/2022 at 6:34 AM

Employee Post

Just chiming in here to provide some context. I had suggested stopping the service and verifying it was down prior to rebooting because I have seen (occasionally) in busier environments the service can take a little longer than expected to come down as it is trying to gracefully disconnect clients. That prevents corruption and the like but my theory was that it may not be terminating before Windows calls it quits and shuts down anyway. So far I've heard this has helped in Jay Dubb's environment, so this is something we can look at trying to reproduce in-house then get it over to development to see what we might be able to do about mitigating it in the future. For the moment, can you set up a script that stops the service, issues a wait, then proceeds to reboot the server normally?

Kyle Kerst IT Manager SmarterTools Inc. www.smartertools.com

Tim Uzzanti Replied

2/17/2022 at 11:11 AM

Employee Post Marked As Answer

Jay,

Kyle hit it on the head, based on the information I'm reading and some other notes. The behavior you are seeing is only from premature termination of the mail process, either manual or by the server reboot, or a crash of SmarterMail. That said, there are no indications of a SmarterMail crash based on what Kyle has evaluated.

I'm not sure why you are rebooting your machine so frequently but if its for SmarterMail, you shouldn't do it.

SmarterMail shutdown time isn't based on us but what is happening with the operating system as a whole. Things like disk i/o, network i/o, terminating server connections, and much more occur when a shutdown is requested. SmarterMail, in most cases, is waiting for timeouts and or actions with other processes and we can't shut down until we get OK's from potentially thousands of different variables.

Anyway, keep your server and SmarterMail running for as long as you can because that is when we run the best. We can't possibly write everything to disk about the current conditions, and that's where memory and things we track overtime are very important for efficiency.

Tim

Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com

Back to Community Threads

Please leave this box unchecked

Reply to Thread

Enter the verification text