Back to Community Threads
24
Feature Suggestion: Modification to Auto-Clean Process #1
Idea shared by Nathan Harrington - 12/17/2014 at 6:05 PM
Completed
At present, the Auto-Clean mechanism appears to base its message deletion decision on the date the message was sent/received (i.e. the message date).  This works well for Inbox, Sent Items, and Junk E-mail; folders in which messages are usually placed at the time they are sent/received.  The Deleted Items folder, however, is a different scenario.

Messages could be, and often are, deleted (moved to Deleted Items) long after they were originally sent/received.  With the current Auto-Clean mechanism, these messages are essentially auto-cleaned immediately (the next time the process runs), rather than waiting the specified period of time as one might expect.  For example, if I'm cleaning out my Inbox and delete a message that I received 6 months ago, that message would be auto-cleaned from the Deleted Items folder later that night, rather than waiting the period of time specified in the Auto-Clean setup.  This behavior renders the Deleted Items folder significantly less useful for recovering messages that were deleted by accident, as there is little time for someone to discover the issue and recover the message prior to it being auto-cleaned.

I would like to propose that the Auto-Clean mechanism base its deletion decision on the date the message was placed in the folder being cleaned.  For the Inbox, Sent Items, and Junk E-mail folders, this isn't likely to result in any signficant change in behavior, aside from the scenario where a message was moved from Inbox to another folder, then at some later date moved back to the Inbox.  For the Deleted Items folder, however, this change would mean the messages would be auto-cleaned X days after they had been deleted (moved to the Deleted Items folder), rather than X days after they were originally sent/received.  This would give people a reasonable amount of time to discover their error and recover the message that had been accidentally deleted.

The ultimate goal (in our case) is to be able to setup Auto-Clean on the Deleted Items folder so that users don't have to remember to periodically empty this folder to prevent their mailbox from getting full.  This was never an issue with POP, but with more and more people converting to IMAP, we are continually running into issues with the Deleted Items folder filling up users' mailboxes.  We have sent several communications to users that they need to perform this additional step, but it remains an on-going issue.  While some mail applications have the ability to perform the purge function themselves, different applications default to different time periods, some default to not being enabled, and some don't provide that capability at all.

We would like to enable Auto-Clean on the Deleted Items folder to help minimize this issue for our users, but with the Auto-Clean behavior as it is now, it's not really an option for us, nor does it seem very intuitive.
 

46 Replies

Reply to Thread
5
Nathan, well said.  This should be considered a flaw that needs to be fixed.
 
If someone deletes a message from last year, it would be natural to assume that they could find it later in Deleted Items.  With the current behavior, it could be gone within hours.
 
Gmail and other mail systems purge messages 30 days after they are placed in the Trash folder, regardless of message date.
 
SmarterTools Team:  Is there a datestamp of when the message is placed in Deleted Items that can be used for purging?  If so, let's change the Auto-Clean behavior to use this date.  If not, I'm sure there are other ways to do it.
 
Thanks,
Kevin
3
Employee Replied
9/28/2015 at 9:29 AM
Employee Post
Thanks for your ideas and comments. This has been added to the developer discussion for a future version. Updates will be posted here when available. 
2
Great, thank you for adding it to the list. This is how other email systems work and it's what users expect, so hopefully the developers will, as Jean-Luc Picard says, make it so.
3
Hi
 
This is really a feature we are missing with 20.000 users on a single domain auto-clean up is a must have feature. Cleaning up deleted mails based on when the mail was received and not when it was deleted does not make much sense. It is very hard to explain users that the mail they deleted yesterday is gone, but the mail they deleted two weeks ago is still there.
 
I really hope you will come up with a solution for this very soon.
5
Hello,
 
Just checking to see when we can expect a fix for this. Obviously it's pretty important based on the number of votes.
 
If there's a technical limitation (like the date deleted is not stored anywhere), let us know and maybe someone in the community can come up with a creative solution.
 
Thanks,
Kevin
4
Happy New Year!  Just wondering if the auto-clean for Deleted Items will be fixed in one of the two new versions for 2016.
 
Here's a possible solution: When a message is deleted, store it in the GRP file with today's date (not the date of the message). Then for the auto-clean, just delete the GRP files that are 30 days old (or whatever time limit you have set).
 
Thanks!
2
I would also like to re-suggest that SmarterTools provide us with a way to handle cleanup of other Deleted Items folders such as "Trash", "Deleted Messages" , etc.  Just because the email client vendors cant agree on what to call the folder, they all have the exact same purpose and we need to be able to clean them up accordingly.
 
6
ST Team, congrats on the release of SmarterMail 15.0!  Hoping to get this on the list for 15.1:
 
  • Fix Auto-Clean so it uses the deleted date instead of message date when purging messages from Deleted Items.
It's got 17 votes (top 5 on forum) and fits well with v15 as it's not related to the UI.
 
Thanks,
Kevin
 
4
Excited to hear that more resources will be dedicated to SM.
We have been slowed a bit because we have so many resources on the new Interface and introducing the new API driven architecture of our products.  What we have done is pulled a significant number of resources from the re-design and moved them over to SmarterMail to ensure we have current versions of SmarterMail as solid as possible.  Expect minor releases over the next few weeks to address most of the concerns.
--Tim U.
Hoping the bugs/problems with the most votes can be addressed. And this should really be classified as a problem, not an idea because it's a flaw in the operation. Thanks for the consideration.
2
Andrea, <bribe> if we can get this item added to the list for v15.1, I will personally deliver a box of Reese's Peanut Butter Cups for the developers. </bribe> Thanks! :-)
6
TGIF. Now that the elephant in the community has been marked as completed :), I'm hoping we can turn our attention to this bug which has almost 2x the votes!
 
Auto-Clean should use the date deleted instead of date received when purging messages from Deleted Items.
 
This is how every major email system works. It's even how other things work, like Apple Photos. When you delete a photo on your iPhone, it keeps it for 30 days regardless of the date you took the photo.
 
We turn Auto-Clean off to work around this bug and give users a fair chance of retrieving a message that they accidentally delete, but this causes other issues so we'd like to see it resolved. Thanks for keeping this item under consideration -- what would it take to flip it to being fixed?
2
Hi Andrea, just checking to see if you have any update on this. With 19 votes, I'm hoping it's on the list for a future version. Thanks.
4
SM has been thinking about this for almost 2 Years. Any hope?
Kendra Support http://www.kendra.com support@kendra.com 425-397-7911 Junk Email filtered ISP
5
Also, would love update.
 
3
For those of you following this thread, check out the discussion with Tim Uzzanti here (scroll down to see the comments under my post): https://portal.smartertools.com/community/a88230/events-with-recurrence-option-are-deleted.aspx
 
Looks like SM understands the problem and is going to work on a fix after ver.16 is released!
1
Is there any chance that ( "Trash" "Deleted Items" "Deleted Messages" ) and ("Sent", "Sent Items", "Sent Messages") could be consolidated into one "trash-type" folder and one "sent-type" folder per user?  
 
That would make all of these cleanup procedures easier to manage for admins and users alike.
 
I previously suggested combining the deleted messages folders in the thread, but it occurs to me that the Sent folder could also benefit from the same logic.
1
Employee Replied
1/26/2017 at 3:42 PM
Employee Post
Hi guys,
 
We can definitely see the value in making this modification. Unfortunately, it's one of those features that sounds simple to make, but has side effects. Changing the auto-clean order requires changes to the GRP files to track not just the message date but the date that the item got moved into this folder. Because of the impact that this change could have, it has been backlogged to a future version where it can be fully engineered to ensure that it has no production server impact. The developers have been putting together a plan to do it, but that change will likely not happen in 16.0.
 
Please stay tuned for updates! I'll post them here as I get them. 
 
Thank you,
1
Andrea,
Thanks for the update. I have to say, this is incredibly disappointing and frustrating news. I reported this issue over two years ago (v12 era). Since that time we have progressed through v13, v14, v15, and soon to be v16 (now confirmed to not be happening in that version). While this may not be as simple as checking a timestamp (i.e. the timestamp needed isn't currently being stored), it doesn't seem like it should take 4-5 major releases of the software to implement this feature. The current method of purging doesn't make any sense at all, especially for Deleted Items, aside from the fact that the message date is already present for use. From what I understand, v16 was a major overhaul of code and expansion of features and capabilities. It seems like it would have been the perfect opportunity for ST to resolve this issue, but they have passed on it yet again. Based on this track record, I don't have a lot of hope that they will ever address this issue. I hope I'm wrong, but seriously, this doesn't seem like it should be that major of an issue, even based on what you have described. Yes, the file structure will need to be changed to add a field for the timestamp, and the auto-purge code will need to be modified to use the new timestamp instead of the message date as it does now. Is it really that big of a deal? It seems ST has little trouble spending a bunch of time developing completely new features, but is unwilling to spend time to fix issues that were reported years ago.
Regards,
Nathan
4
Yes, understand that the deletion date doesn't get stored in the GRP file. Here's an idea from an earlier post that might provide a workaround, FWIW:
For the Deleted Items folder only, store messages in the GRP file with the date the message was deleted (not the date of the message). Then for the auto-clean, just delete the GRP files that are 30 days old (or whatever purge timeframe you have set).
7
It's been over a year since the last post, so I'm following up. This fix was supposed to be looked at after v16 was released, but it doesn't look like anything has been done. Just checking to see if it's on the list for v17.
Currently, messages are purged based on received date, not deleted date. So if a user accidentally deletes an old message, it could be immediately purged from Deleted Items. They should stay in Deleted Items for 30 days like they do in Gmail and other email systems.
It is one of the top 10 most-voted-for ideas on the forum. Thanks.
4
So I just wanted to add my vote up to this discussion. I just had client which removed his whole Inbox and his messages where removed from deleted folders almost right away:
 
[2018.04.17] 11:53:17 [CLIENTEMAIL] Auto clean options did not meet cleaning criteria
[2018.04.17] 11:53:18 [CLIENTEMAIL] Starting auto clean
[2018.04.17] 11:53:20 [CLIENTEMAIL] 949 total affected groups list
[2018.04.17] 11:53:20 [CLIENTEMAIL] Completed auto clean
[2018.04.17] 11:53:20 [CLIENTEMAIL] Removed 5497 email(s) from folder Deleted Items
[2018.04.17] 11:56:26 [CLIENTEMAIL] Auto clean options did not meet cleaning criteria
as you can see at 11:53 where was no emails to be removed and few seconds later auto cleaning job has removed 5497 of accidentaly deleted messages. There was literaly no time to get them back from Deleted items folder (global auto cleaning rule is removing messages older than 30 days).
 
This has happened for me probably multiple times but then I didn't connected messages dissapearing with auto cleaning process and just assumed something bad happened and messages where just removed because of some bug.
2
Webio, good example! This is the point I've been trying to make.
2
Somehow users manage to delete all the messages in their SmarterMail Inbox on a regular basis. This is not good because most users have thousands of messages in their Inbox.
 
One option is to turn off auto-clean completely. It's not a good solution because it causes other issues, like full mailboxes and low disk space.
 
I'd really like to know how users are deleting their whole Inbox.  Not sure if it's a bug or they are using the "Delete All Messages in Folder" menu option or the top checkbox that selects all messages. Would be nice to add some code to make deleting all the messages in Inbox more difficult.
4
So ... any news with this feature? I'm still on latest 15.x waiting for 17.x and I had similar problem like before. Client moved his messages from Sent folder to Deleted items and they got removed right away.
0
Employee Replied
9/26/2018 at 1:22 PM
Employee Post
Hello all,

I wanted to copy over Tim's reply from a thread in the SmarterMail 17.x BETA category: "This feature won't make it into 17 but because the entire back-end has changed it will now be much easier for an upcoming release."

Kevin, I've also put in a request to make it harder / more obvious when a user is deleting all items in a folder. 
2
Can you clarify how we should understand upcoming? When you say it will not make into 17 you mean general availability 17 or not a chance for any 17.x version and we will have to wait for 18?
0
Employee Replied
9/27/2018 at 12:42 PM
Employee Post
Hi Webio,

I'm afraid we don' have a tentative release date scheduled at the moment. I will provide updates as more information is discussed / planned. 
1
Ok but having information 17.x or 18.x is quite crucial here.

I had another similar issue with moving messages to deleted folder and cleaned it right away by auto cleaning process.

EDIT: Maybe some work around for this problem in current versions would be allowing to set as mail server admin how often auto clean should be performed or maybe during what day and hour? So I would set it to sunday about 6AM or something similar.
2
Webio, that's a good idea for a workaround until the proper solution can be implemented.

Let's include the day of week also for the autoclean. Then I could schedule it after the full backup that runs weekly.
2
So .. now when v17/v100 is released can we return to this topic (I presume that in v17 it still works like it was described by many in this topic)?
2
@Webio, I see you referred to this problem in a recent thread.

Ironic that the last post here was almost exactly one year ago. As far as I know, this still does not work like it should. Our customers would really like to see it fixed. Evidently with 22 votes, so would everyone else...
3
+1 again...
4
my +1 here as well
Sébastien Riccio System & Network Admin https://swisscenter.com
5
We had a customer accidently delete messages from their inbox today. The older messages were then immediately removed from the Deleted Items based on the Folder Auto-Clean rule. It would be nice if the Auto-Clean feature could be at least improved to run once a week or month. Thanks!
3
Yes, please! :-)
5
Scott, good bump on this old thread.  Wow, today is the 6-year anniversary of this post. Wonder if Nathan is still using SmarterMail. :)

Anyway, you provided a great reason why it's so important this gets fixed. Almost every email service purges deleted items 30 days AFTER you delete the message. Not based on the date the message was received. If you delete a message from 2 months ago, it shouldn't be gone tomorrow.

Looking back I provided an idea for fixing this:
For the Deleted Items folder only, store messages in the GRP file with the date the message was deleted (not the date of the message). Then for the auto-clean, just delete the GRP files that are 30 days old (or whatever purge timeframe you have set).

Hope this gets changed from Planned to In Progress for 2021. Thanks!
4
+1
Gabriele Maoret - Head of SysAdmins at SERSIS Currently manages 6 SmarterMail installations (1 in the cloud for SERSIS which provides services to a few hundred third-party email domains + 5 on-premise for customers who prefer to have their mail server in-house)
5
Would really like to see this addressed. Just yesterday I had a user accidentally delete their inbox and all the old messages were gone instantly. Had to do a restore.
6
@kevind No, he is not still using SmarterMail, however it has nothing to do with issues such as this.  Changes to our business dictated that we move to a different mail solution.  It isn't perfect either; in some ways better, in some ways not as good, but it does what we need it to do within the available budget.

In general I have enjoyed working with the folks at SmarterTools over the years; most of them have been very helpful.  We have used several of their products and in general have been happy with them.  I would still recommend SmarterMail for consideration in situations where it fits the need.

That said, I gave up on SmarterTools fixing this issue long ago although I do find it interesting that this thread keeps popping up after all these years.  Despite this request being among the top five suggestions over the past six years, it has received no meaningful attention from SmarterTools.  It continues to get put off despite several major version releases including at least one major rewrite that allegedly would have allowed them to address it.  I'm not in their shoes, so I don't know everything that is involved in fixing it, what competing priorities there may be, etc.  I know it was a big issue for us, and one that was never addressed, but perhaps we are the minority with respect to their overall customer base despite the popularity of this suggestion on this forum.  Regardless, this sort of functionality seems to work in a logical, more useful manner with other mail platforms I use so I still feel that it's something SmarterTools should address eventhough we aren't currently using SmarterMail.
2
Tim Uzzanti Replied
1/14/2021 at 4:08 PM
Employee Post
All,

This feature is available in our next release.  Adding this required changes to our backend data which we very much limit how frequently we modify.  Please refer to the links below for more information:

User: 

Domain: 

Hope this helps.
Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com
2
Hello Tim,

This is good news. Has it be well tested on some big systems before adding it to the release?
This is a sensible component. We're waiting for the next release to fix the other bugs and wouldn't like that a bug introduced by changes in auto cleaning make it starts to wipe our customer e-mail and having to restore them.

Sorry to be pessimistic, but I prefer to be careful about these kind of changes because we have already enough other issues without adding a new one.
Sébastien Riccio System & Network Admin https://swisscenter.com
2
Awesome news on this feature!

@Sébastien Riccio With the "We changed the backend" notice, we will be delaying the install :), just as we did with the folder translations which allowed us to dodge a big bullet.
3
Well echoDreamz, that's a bit why we are being careful here. But well on the other side we can't really delay the install because we are overflowed with supports of angry customers about issues that should be resolved in this coming build, so we MUST take the risk :/

That's why we asked/suggested many times two different branches. One stable with only bug fixes and another more beta for testing some big changes or new features instead of adding them directly in "stable".

Doesn't seem that it will go this way anytime soon though
Sébastien Riccio System & Network Admin https://swisscenter.com
2
I agree with this release type. We have this with another product we use. Stable which is only bug fixes, security updates and does not include anything "new" unless they are minor changes. Then they have a rolling release which includes all the bug fixes etc. from stable + shinny things that will eventually get rolled into stable.
2
This is great news! Our customers will be happy not to lose important messages and this will help us save money by freeing up disk space.

Our MO: We have a test server with the free version that we upgrade first. Then after a week or so we'll upgrade the production servers. We rarely have an urgency to upgrade right away.
0
Glad to have this feature and hope the "backend" change doesn't introduce new bugs. Like Sébastien we need to update to get bug fixes, so we'll be installing the new build soon.
Back to Community Threads

Reply to Thread

Related Knowledge Base Articles

New SmarterMail and WHMCS IntegrationUtilities and Conversion Tools
Maintenance and Support - Renewal / ReinstatementGeneral Topics
Recommended SPAM SettingsSmarterMail > Antispam and Antivirus
SmarterMail Server Performance TuningSmarterMail > Server Configuration and Management
Deploying Rspamd For Use With SmarterMailSmarterMail > Server Configuration and Management