SmarterMail 14.5.5907 huge GRP file

Problem reported by Jay Altemoos - 4/5/2016 at 12:38 PM

Submitted

I had a user call in today with a slow webmail issue with SmarterMail 14.5.5907. So I checked the users mailbox and sure enough when I either impersonate that user or login with their credentials webmail is slow to populate the Inbox only. So I connected into the server directly and found that their GRP file for today is 1.3GB in size.

So I ran a report in SmarterMail and found this is the delivery logs for the past several hours:

[2016.04.05] 15:28:47 [48783] Starting local delivery to user@localemail.com
[2016.04.05] 15:28:47 [48783] Add message to mailbox call failed. Reason: Access to the grp file is locked
[2016.04.05] 15:28:47 [48783] Delivery attempt to user@localemail.com to folder 'Inbox' failed. Mailbox is locked.

Now we are running anti-virus on the mail server and I do have a rule setup to exclude all SmarterMail folders. This includes the actual user folders along with the program directory. Once I restarted the IIS 7.5 service then webmail populates fine. I also found that webmail for all users was slow before I restarted the IIS service. So this one user was slowing down the entire web service all because the GRP file was so huge? I verified this by logging into another machine and tried going to our webmail landing page and it just sits there while the user with the huge GRP file was logged into webmail attempting to view their Inbox. Only way I have been able to correct the slowness was to restart IIS and then everything is good again. That is until this user with the huge GRP (1.3GB) file logs in again and tries viewing their Inbox. Then the process repeats itself.

Checking through the taskmanager on the server itself here's the memory usage for both IIS and SmarterMail:

w3wp.exe 605MB and mailservice.exe 1.2GB

We have 16GB of ram installed on this box so memory is not the issue. We are also running a Intel core I7 processor. Both memory usage and CPU usage have been normal even through the issue I stated above. I have seen this same exact situation happen about a month or so back with another user that had a huge GRP file. The only fix I could find at the time was to remove the GRP file. I don't want to keep doing this and have users lose important email all because the file size of a GRP file got too big. Compound this by several users and I have a bunch of angry users. When I looked at the index status in SM it was stuck trying to index that entire 1.3 GB GRP file for this user and appeared to not show any progress. It though only 1 email had to be indexed and I am sure there was more in that file than just that.

Any one else run into this issue? If so what was the fix for it?

Also while we are on the subject of GRP files, any plans for the future to have large GRP files broken up into smaller chunks instead of lumping it all into one file for that day? Basically right now I have a user that can't use webmail because their session ends up holding up the entire web interface for everyone we host all because the GRP file is too big. Eventually the user with the huge GRP file will get the "Ooops something went wrong message" if they wait long enough trying to view their Inbox. Which appears to crash the web interface for them. Everyone else just has to wait several minutes before they get the log in page for SM and it's slow to log in. Restarting IIS resolves this but seems like a band aid for the overall problem. The GRP gets too big and SM has a hard time handling it properly.

4 Replies

Reply to Thread

Scarab Replied

4/5/2016 at 1:13 PM

Jay,

I have not experienced this problem and although we are on the same version of SmarterMail I double-checked our user's GRP files and found that the largest GRP file we have for even our heaviest of users is 27MB and the average is 4-6MB.

Have you tried running an Rebuilding the Folder of that user's account rather than restarting IIS? It sounds to me like data corruption and SmarterMail is hanging up on attempting to Index it (which is why you are getting the error "Add message to mailbox call failed. Reason: Access to the grp file is locked"). My suggestion is to try to rebuild their Folder (MANAGE > RESTORE).

This kind of corruption can generally be prevented by more frequent Indexing. What are your Indexing Settings (SETTINGS > GENERAL SETTINGS > INDEXING) in SmarterMail?

Scarab Replied

4/5/2016 at 1:59 PM

Follow up: I was wrong. We have 3 users with GRP files that are @ 3GB in size each (all of them \Drafts). Confirmed in all three cases they were data corruption (and all 3 accounts were IMAP users).

After trying in vain to purge deleted items in the \Drafts folder we had to stop the Indexing Service in MANAGE > SERVICES and then use MANAGE > RESTORE > REBUILD FOLDER on those folders and restarted the Indexing Service. The Indexing and Optimization passes took a while, but after it was complete they were back down to 300KB - 3MB each.

Jay Altemoos Replied

4/5/2016 at 2:17 PM

Here's what I got for Indexing:

Max Threads   6
Segment Count Before Optimizing   20
Items Before Garbage Collection   5000
Items To Index Per Pass   2500
Seconds In Queue Before Indexing   5
Deleted Items Before Optimizing    1000

These settings have been in place since we installed SM 3 years ago. I did how ever uncover one of the issues with the users mailbox. As it turns out one of the co-workers kept sending a 70mb MOV file to this user and sent it 15 times. So 1GB worth of information was this one user alone. Still the issue remains that when a GRP file gets this big, there's a chain reaction that happens that is not good. Basically bringing web mail down to it's knees. I really think a rule in SM could be set to limit GRP file size to break the file out into smaller chunks. Then issues like this could be avoided. While I haven't run into this issue a whole lot, as I add more users to our mail service I can gather I might run into this further. I guess I could limit down file size attachments, but this is something we haven't restricted on our users. It might be something our company needs to revisit.

Jay Altemoos Replied

4/6/2016 at 6:55 AM

So finally after 4 hours of SmarterMail indexing this huge GRP it finished. So for that length of time my user could not access their Inbox via web mail because it would time out the web interface. Then the chain reaction from that would time out the web interface for all my users and I would have to restart IIS to get the web interface active again. Even IMAP for this same user was slow to download each of those files. For now I instructed those 2 users to not send such large attachments through email. One of our selling points for our customers is that we don't limit them on email attachment sizes. Most of our users are good about not sending monster sized attachments to each other. In this case it appears this person's iPhone kept sending the same email over and over again. Which in turn caused the GRP file to grow to 1.4GB and slowed down the index for SM to a crawl.

Just an idea here that I will post in the suggestion section would be to separate the attachment from the email somehow so the GRP file doesn't grow that large. Maybe a attachment folder that is linked to each email somehow. Maybe message ID, etc.?

Back to Community Threads

Please leave this box unchecked

4 Replies

Reply to Thread

Tags

Related Knowledge Base Articles