Hi Sebastien, I checked my POP and IMAP logs. Until June 3, there was some error similar to what you report, but I found them only in the POP log.

From June 4 onwards (after I made the update to the official version 7459) I no longer report even one of these messages.

I don't know if this can help you ...

Hello,

Thanks for your feedback.

In our case on 20k active accounts the error is popping in imap and pop for around 200 accounts and they match the users having reported imap instabilities and some have "This message has been removed" messages in their inbox.

We updated to 7459 the night it was released, but the errors still occurs.

I think deleting *.cfg and rebuilding the mailbox will fix it but I would like to be sure doing so has no other impact on the mailbox and having these customers opening ticket for other strange behavior, like undeleted messages.

That's why I would like to understand what are in these .cfg files and what is their purpose :)

Sébastien,

The mailbox.cfg contains a list of all emails and associated properties and flags within a particular mail folder. It will contain the UID, MID, message flags, last used UID (for IMAP), and other "quick" information SmarterMail will use in place of having to load all of the GRPs and parse that information (very expensive).  In case that SmarterMail does not detect a mailbox.cfg, it will rebuild it from the GRP files. However, it will lose message flags, etc.

The deleted.cfg file is no longer used and should be cleaned up over time.  I will add a task for us to investigate why these are not being cleaned up.

Hello Robert,

Thank you for these informations. 

I'm still investigating how to recover from the corrupt GRP error messages we have (they seems to be linked to users having "This message has been removed" in POP and/or IMAP)

1) So about the deleted.cfg it's safe for us to remove them from the users forlders ? I've checked quite a lot of folders and they all have their deleted.cfg file present...

2) Removing the mailbox.cfg to have SM recreate it will then lose the status of the e-mails (replied flag, etc?).

This is quite a problem because we already have cusomters opening support case telling us that replied messages have lost the reply flag, so they have to check the sent items to be sure if they replied or not...

I've ran a little script over our logs to determine which folders are impacted:

#!/bin/bash
grep -hioP '[A-Z]\:\\SmarterMail\\Domains\\.*\\(?=.*\.grp)' 2020*log | sort -u > corrupt.log

The logs shows the moment 252 "unique" folders being affected by this:

$ wc -l corrupt.log
252 corrupt.log

But it seems every day around 10 more unique corrupt folder is detected. 

So either the problem is propagating or maybe the new folders are users that haven't accessed their mailbox since we upgraded to latest beta release, therefore not generating any log entry. I'm not sure what to think here.

About the flags being lost, isn't there a copy of the flags in the GRP files that it can grab when reconstructing mailbox.cfg ? Or is there an utility to clean the mailbox.cfg and fix without losing the flags ?

It's important for us that when attempting to fix these corruptions errors, we do not add problems such as these flags being reset...

ps: We have a ticket open for this issue (1E3-266A74AE-0B0D)

Kind regards.

1. You can safely delete the deleted.cfg files. We have a task to cleanup those files.

2. The GRP file has a header section for each message that only includes the message UID, its start offset in the file, and its end offset (to calculate length).  It does not contain any other message information.  That information is solely contained within the mailbox.cfg file. If the mailbox.cfg is recreated, it will have lost current message flags.

Hello Robert,

Thank you for your reply.

So basically removing mailbox.cfg to clear a "corrupt grp" issue will reset all the messages status, like being read, being replied, etc ?

We can't do this, our customers will open tickets if their messages status are reset...

So this bring us again to the question. Is there a way to fix the mailbox.cfg files for these "corrupted" mailboxes without losing message states/flags ?

Like removing/clearing from the .cfg only the "corrupted" data that leads to these "corrupted GRP" log entries and also "This message has been removed" issue the customers reported ?

I hope you understand we can't just remove the mailbox.cfg and lose the message flags that are valuable to our customers!