Missing Messages and GRP indexing errors

Problem reported by Scot Desort - 3/14/2018 at 4:36 PM

Submitted

Running 15.5. ONE user started to receive those weird messages from Smartermail on Sunday:

Subject: Message removed

A message has been removed from this mailbox by an entity other than this program, probably by a virus scanner.
This message is a replacement for the missing message.

And it has happened every day since then up until yesterday. We double check that the on demand OS virus scanner was excluding the folder.

After that (yesterday), it seemed OK. Today the user emails me and says he hasn't received email in an hour an a half which is very strange. I check the delivery logs and I see messages coming in, but in the IMAP and POP logs, I don't see messages being read/retrieved. I then indexed his box. Then I checked the indexing log, and stumbled across this for today (prior to my manual index request):

[2018.03.14] 16:20:02 [xxx@xxx.com] Error during indexing, invalidating index: System.IO.FileNotFoundException: Could not find file 'c:\SmarterMail\Domains\xxx.com\Users\xxx\Mail\Inbox\2018_3_14.grp'.
[2018.03.14] File name: 'c:\SmarterMail\Domains\xxx.com\Users\xxx\Mail\Inbox\2018_3_14.grp'
[2018.03.14]    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
[2018.03.14]    at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
[2018.03.14]    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
[2018.03.14]    at MailStore.GroupFileAccess.GroupFile.LoadHeader(GroupFileItem messageBoundary)
[2018.03.14]    at MailStore.GroupFileAccess.MimeReader.LoadHeader()
[2018.03.14]    at MailStore.GroupFileAccess.MimeReader.get_Header()
[2018.03.14]    at MailStore.Indexing.Indexables.IndexableEmail.#Pkb()
[2018.03.14]    at MailStore.Indexing.Indexables.IndexableEmail.GetDocument()
[2018.03.14]    at MailStore.Indexing.UserIndexer.IndexUser(UserIndexItem userIndexItem, IndexingContext context, String username, String domainName, PendingIndexDeletions pendingDeletions, Log log)

I then went further back to Mar 1 and saw the same thing on 3/11.

No other user in this domain or any other domain on this server is reporting an issue.

This user IMAPs his email on an iPhone then POPs the box from his desktop (if that helps). Appreciate any assistance.

Scot

11 Replies

Reply to Thread

Matt Petty Replied

3/14/2018 at 4:59 PM

Employee Post

Hello,

This was something that we fixed recently in 16, however it looks like this change did not make it into a minor of 15. I will make this change immediately and send you a link to a custom build which contains a fix for this issue.

EDIT:
Sorry, it was something else but very related to this issue. Evaluating the issue further and it is not the cause in this case. However, I think I do see the issue, I'll get back shortly...

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Matt Petty Replied

3/14/2018 at 5:19 PM

Employee Post

NOTE To Scot: Don't do this, I might have a fix for you shortly... However, I'll keep this post here for anyone else debugging a issue like this one.

Hello Scot,

The indexing system is expecting a certain message to exist when it does not and it's trying to access the file that used to have the message. This error is causing the indexing to loop on itself, since if it gets an error indexing, it will then re-index the user again, rinse and repeat. this odd behavior is part of the problem but is not the root cause.

You'll have to do this while the SmarterMail Service, MailService, is turned off.
Could you try visiting the following folder and removing the "mailbox.cfg" file inside of it?

"c:\SmarterMail\Domains\xxx.com\Users\xxx\Mail\Inbox\"
While your near this area could you step back a directory and delete the "Index" folder, this will cause the user to be re-indexed and also clear out anything I has already indexed.

After this, boot up SmarterMail again and it should recreate this file.

NOTE: This is the file we use to store message flags in, which will cause any flags on a message in that folder to be lost, replied to, forwarded, unread, etc. If this is a concern for you, I'd back this file up in another location first.

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Matt Petty Replied

3/14/2018 at 5:55 PM

Employee Post

Scot, I sent you a build with a fix in place for this issue.

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Scot Desort Replied

3/14/2018 at 5:59 PM

Thanks for responding so quickly, Matt. Is this fix also present in the latest 15.x release, Version 15.7.6638 - because I can easily install that.

If not, what minor revision is your fix based on ?

Thanks again

--
Scot

Matt Petty Replied

3/15/2018 at 7:12 AM

Employee Post

The build I sent you is 15.7.6647, which is about 9 days newer. This was made yesterday which contains any fixes since that version, I don't think there have been many other fixes to 15. We have been very busy with SmarterMail 17. I just checked and the other fix in the build I gave you fixes a boot up issue if SmarterMail fails to load a user.

If you go to "My Profile" from the top right on the community after clicking on your name, you can see the private messages from me. I just sent a link, you can use that download.

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Scot Desort Replied

3/15/2018 at 10:22 AM

Thanks. We are scheduled to install it tonight.

Yesterday, I went into the user's box and set it up to forward a copy of all email to another box. Now THAT box shows an index error, and the user just emailed and said he is again missing another hour of messages and got that MESSAGE DELETED email (on his original box).

I don't understand why a completely different mailbox, brand new, is now having the same issue. My guess is it's related to content in an email that he is getting every day that SM is puking on ? Am I close?

Matt Petty Replied

3/15/2018 at 1:08 PM

Employee Post

I'm going to open an outbound ticket with you, I'd need your indexing logs since you made the change above. Then I'll need the 2 email addresses. Just send this info in the ticket, it will be going to your email address that you registered on the community with.

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Matt Petty Replied

3/15/2018 at 1:12 PM

Employee Post

But to answer your question yes, you are very close with your assumption. In fact the first reply I made, the one that I crossed out, was an issue we experienced ourselves here at the office with a user.
They had a field that gets indexed, like To:, Except the data in this To: field was so full of recipients the indexer gave exceptions. It could be similar, this fix was never rolled down into 15.

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Scot Desort Replied

3/15/2018 at 1:17 PM

Matt

The fix has NOT yet been installed. it's being installed tonight. So the logs I have currently are pretty much the same as the ones you saw already above.

I am now forwarding copies of his email messages to a non-SM mailbox on another server so at least he won't miss anything important until the update is installed tonight. I will circle back on this tomorrow and let you know how things are going.

Matt Petty Replied

3/15/2018 at 1:21 PM

Employee Post

What we'll wanna look for is the first exception occurring in the indexing logs for the new user you had setup. The first exception we see coming from that new user should give us hopefully a pretty tell-tale exception message/trace that can help.
Just send me the indexing logs in that new ticket for today and yesterday and the 2 email addresses and I can look for that myself.

Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com

Scot Desort Replied

3/19/2018 at 4:41 AM

Matt-

So far, so good. This appears to have resolved the issue. No reports from the end user of any "MESSAGE DELETED" emails or missing emails. Thanks again for your prompt response to this issue.

Back to Community Threads

Please leave this box unchecked

11 Replies

Reply to Thread

Tags

Related Knowledge Base Articles