What are the use of mailbox.cfg and deleted.cfg
Question asked by Sébastien Riccio - 6/5/2020 at 10:42 PM
Unanswered
Hi,

Since we updated our production server to mapi release, we have a significant amount of:

00:06:19.661 [x.x.x.x][48826906] RETR failed to FeedMessage for message 1 (UID: 609) ERR: The group file F:\SmarterMail\Domains\xxx.ch\Users\xxx\Mail\Inbox\2020_6_4.grp is corrupt or a message could not be found in the header..

in both pop and imap logs.

We have an open ticket about this and at this point the solution seems to be to remove *.cfg from the folder and rebuild it.

I tried on a mailbox and it seems it does the trick. So I wrote a powershell script to grab all errored folders (based on log entries), remove *.cfg files and issue a folder rebuild (via API).

However, before batch running the script on all errored folder I would like to understand what are the contents of the mailbox.cfg and deleted.cfg to know what could be side effects of deleting them.

The problem is that these files have binary content and I can't find an utility to view/edit them in a human readable format.

I suspect the problem we have is maybe linked to deleted.cfg and would like to know what is it's purpose.
For example if it keeps track of deleted e-mails, if we remove this file, will SM lose track of the user e-mail flagged as deleted in this folder ? This would result in customers opening tickets about their deleted mails being... undeleted.

As sysadmins having to deal/fix issues, I think we really lack some important checking tools and informations about these file formats.

Anyone can shed some light on this ?

Thanks a lot.


Sébastien Riccio
System & Network Admin

6 Replies

Reply to Thread
0
Hi Sebastien, I checked my POP and IMAP logs. Until June 3, there was some error similar to what you report, but I found them only in the POP log.

From June 4 onwards (after I made the update to the official version 7459) I no longer report even one of these messages.

I don't know if this can help you ...
0
Sébastien Riccio Replied
Hello,

Thanks for your feedback.

In our case on 20k active accounts the error is popping in imap and pop for around 200 accounts and they match the users having reported imap instabilities and some have "This message has been removed" messages in their inbox.

We updated to 7459 the night it was released, but the errors still occurs.

I think deleting *.cfg and rebuilding the mailbox will fix it but I would like to be sure doing so has no other impact on the mailbox and having these customers opening ticket for other strange behavior, like undeleted messages.

That's why I would like to understand what are in these .cfg files and what is their purpose :)
Sébastien Riccio
System & Network Admin

1
Robert Emmett Replied
Employee Post
Sébastien,

The mailbox.cfg contains a list of all emails and associated properties and flags within a particular mail folder. It will contain the UID, MID, message flags, last used UID (for IMAP), and other "quick" information SmarterMail will use in place of having to load all of the GRPs and parse that information (very expensive).  In case that SmarterMail does not detect a mailbox.cfg, it will rebuild it from the GRP files. However, it will lose message flags, etc.

The deleted.cfg file is no longer used and should be cleaned up over time.  I will add a task for us to investigate why these are not being cleaned up.
Robert Emmett
Software Developer
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Sébastien Riccio Replied
Hello Robert,

Thank you for these informations. 

I'm still investigating how to recover from the corrupt GRP error messages we have (they seems to be linked to users having "This message has been removed" in POP and/or IMAP)

1) So about the deleted.cfg it's safe for us to remove them from the users forlders ? I've checked quite a lot of folders and they all have their deleted.cfg file present...

2) Removing the mailbox.cfg to have SM recreate it will then lose the status of the e-mails (replied flag, etc?).
This is quite a problem because we already have cusomters opening support case telling us that replied messages have lost the reply flag, so they have to check the sent items to be sure if they replied or not...

I've ran a little script over our logs to determine which folders are impacted:

#!/bin/bash
grep -hioP '[A-Z]\:\\SmarterMail\\Domains\\.*\\(?=.*\.grp)' 2020*log | sort -u > corrupt.log
The logs shows the moment 252 "unique" folders being affected by this:

$ wc -l corrupt.log
252 corrupt.log
But it seems every day around 10 more unique corrupt folder is detected. 
So either the problem is propagating or maybe the new folders are users that haven't accessed their mailbox since we upgraded to latest beta release, therefore not generating any log entry. I'm not sure what to think here.

About the flags being lost, isn't there a copy of the flags in the GRP files that it can grab when reconstructing mailbox.cfg ? Or is there an utility to clean the mailbox.cfg and fix without losing the flags ?

It's important for us that when attempting to fix these corruptions errors, we do not add problems such as these flags being reset...

ps: We have a ticket open for this issue (1E3-266A74AE-0B0D)

Kind regards.

Sébastien Riccio
System & Network Admin

0
Robert Emmett Replied
Employee Post
1. You can safely delete the deleted.cfg files. We have a task to cleanup those files.
2. The GRP file has a header section for each message that only includes the message UID, its start offset in the file, and its end offset (to calculate length).  It does not contain any other message information.  That information is solely contained within the mailbox.cfg file. If the mailbox.cfg is recreated, it will have lost current message flags.
Robert Emmett
Software Developer
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Sébastien Riccio Replied
Hello Robert,
Thank you for your reply.

So basically removing mailbox.cfg to clear a "corrupt grp" issue will reset all the messages status, like being read, being replied, etc ?

We can't do this, our customers will open tickets if their messages status are reset...

So this bring us again to the question. Is there a way to fix the mailbox.cfg files for these "corrupted" mailboxes without losing message states/flags ?

Like removing/clearing from the .cfg only the "corrupted" data that leads to these "corrupted GRP" log entries and also "This message has been removed" issue the customers reported ?

I hope you understand we can't just remove the mailbox.cfg and lose the message flags that are valuable to our customers!

Sébastien Riccio
System & Network Admin

Reply to Thread