5
Data DeDuplication with Mail
Question asked by Tim Walker - 6/5/2019 at 9:16 AM
Unanswered
Has anyone in the community enabled Data DeDuplication for their smartmail server? 

Wondering what kind of compression you get, and whether it has negative effects on performance?

Let me know

Thanks,

Tim

7 Replies

Reply to Thread
2
Kyle Kerst Replied
Employee Post
I am curious to see what kind of responses we get from the community on this one as well! I have recently deployed some deduplication testing, but with my environment being largely recently generated test messages and sample file uploads, I'm not seeing much of anything in the way of storage gains. Please briefly outline your deployment and any exclusions if you don't mind as well. Thanks everyone!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
1
Kyle Kerst Replied
Employee Post
Quick update for you on this Tim! I just checked on the deduplication dashboard on my test server, and it looks like it just took 24 hours or so before dedup kicked in: 


I set mine up to ignore GRP files just to be safe, and so this is strictly file storage being saved here. My file storage accounts hold about 120MB of misc zip files, images, and text documents I uploaded for good measure. SmarterMail is reporting 120MB in size for this domain, but the dashboard reflects ~62MB total on the drive. I'll continue to monitor it over the next few days while I upload additional files and I'll let you know what I find out! So far so good!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
Kyle Kerst Replied
Employee Post
After some additional checking I am finding the deduplicated copies of files I've uploaded to file storage are no longer present, or are 0KB files. Might need to do some more testing on this before I recommend it to anyone.
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
1
Kyle Kerst Replied
Employee Post
Update as of this afternoon: I'm now seeing the file sizes again, so this may have been a temporary indexing issue. So far 16% savings on my limited file testing!
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
I've been experimenting with Windows Server deduplication on the disk used for SmarterMail storage for several months now.

From what I could see, everything works fine and saves a lot of space!

In my case, it went from 1.9TB of occupied disk space without deduplication to about 1000GB with deduplication on.
Almost half!!!
Gabriele Maoret - Head of SysAdmins at SERSIS Currently manages 6 SmarterMail installations (1 in the cloud for SERSIS which provides services to a few hundred third-party email domains + 5 on-premise for customers who prefer to have their mail server in-house)
0
Manuel Martins Replied
Hi,
We are using Compression and Deduplication on QNAP Storage connect to the SM Server by iSCSI and the savings are about 7-8 %

0
Hi Manuel.

Perhaps the algorithm used by Windows Storage Deduplication is more efficient than the one used by Qnap.

It seems strange to me that on 2TB of data I see a 50% gain and on 11TB you see only 7-8%...

NOTE: As far as I know, Windows Storage Deduplication uses a combination of data deduplication and compression.
Gabriele Maoret - Head of SysAdmins at SERSIS Currently manages 6 SmarterMail installations (1 in the cloud for SERSIS which provides services to a few hundred third-party email domains + 5 on-premise for customers who prefer to have their mail server in-house)

Reply to Thread