1
will SM enterprise work with 25000 or more mailboxes in a vm
Question asked by Bilal Mahmood - 8/14/2024 at 11:24 AM
Unanswered
we will have 64TB nvme windows storage services available for VM via iSCSI. 1TB DDR5 ecc ram. about 48 cpu cores at 3.5Ghz with turbo hitting 5Ghz. system using intel x710T4 4 port network card with all 4 ports bound to ONE with 10Gbps LAN and 2Gbps up and 2Gbps down internet.
do you think this VM can run 25000 mailboxes?

11 Replies

Reply to Thread
0
Matt Petty Replied
Employee Post
Yea, thats a pretty beastly machine. I would say it can, what protocols are you intending to have the users connect with? Some of the protocols are more stressful to the server than others. Also are these going to be on seperate domains or all on one domain. We're trying to optimize the single-domain performance right now. If they are spread across domains it scales way better in our system atm. You could submit a sales ticket and give us a bit more detail if you'd like as well and we could give you a much more informed answer. 

But with a rig like that, I don't think you'll have issues.
Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com
0
Nageswara Rao Anumolu Replied
We are struggling with 15000 users in a single domain with occasional freezing and json file deletions. 
0
Bilal Mahmood Replied
we will be using tcp/ip v4 iMAP and SMTP. We will use built in sniffer and antispam add ons. we will not use Active sync (EAS). we will NOT be using Chat xmpp. purely this will be a mail and calendars server mostly used via Android mobile devices with Multi Factor Authentication.
Our 64T storage will all be SSD (SATA or even possibly NVME pcie) to have millions of iOPS available to SM to have resources for disk operations.
0
Scarab Replied
We migrated away from running SmarterMail in a VM when we discovered that I/O, Wait Queue Length, and Latency were causing a myriad of errors and performance issues despite CPU & RAM usage remaining low. You'll want to make sure that your IOPS, especially for write operations, is more than sufficient as almost all Hypervisors have increased latency involved with VM IOPS, granting a real-world usage @ 25% the IOPS that would be available to the Host if it were bare-metal (many Hypervisors allow you to fine-tune the IOPS per VM but there is still going to be a significant hit in latency and overall IOPS available to the VM).

If I had to do it all over again to run SmarterMail in a VM I would have planned for RAID-10 SSD arrays (avoiding RAID-5 or RAID-6 or even ZFS RAIDZ1 - Z3) and would make certain to use a RAID Controller with a built-in Battery Backup to handle any Writes left in the Wait Queue when power loss is ever encountered.

But other than being mindful of IOPS you should be in overkill territory for CPU & RAM usage for POP3/IMAP/SMTP & WebDAV/CALDAV for 25K+ users in a VM.
0
Jereming Chen Replied
Employee Post
That is a fair assessment since there is just so much moving data, IOPS end up being the biggest bottleneck over most other hardware specs as you scale up. You might benefit from having an SSD cache for your storage if you would still rather go with the more typical RAIDS but I would still agree that a RAID 10 of SSDs would be the ideal storage setup for a SmarterMail application of this magnitude.
Jereming Chen System/Network Administrator SmarterTools Inc. www.smartertools.com
2
That is what we use and we guarantee a minimum number of IOPS for every customer.

But using SATA is a bottleneck since its only 6gbit/s.
0
Bilal Mahmood Replied
Good point about data being inefficient while data throughput be 500MBps*24 however iOps will only be 10,000*24 which is lower than one Pcie ssd
So definitely will use nvme and not sata
2
Jay Dubb Replied
On a system with just under 2,000 active mailboxes, most using Outlook/MAPI and EAS/Mobile, with plenty of webmail users too, we had to buy a new Dell server with 8x Enterprise SAS SSDs (24 Gbps SAS) on a controller with fat cache to keep up with the disk load.  It was ridiculously expensive but the only way to keep disk IO from grinding everything to dust.  Server 2019, Hyper-V, VM with 96 GB RAM and 22 CPU cores.  RAM and CPU have an easy life, but disk IO is always heavy.
 
1
Bilal Mahmood Replied
each nvme ssd on pcie4 using 4 channels with each channel hitting 2GB/s meaning 4 channels x 2GBps = 8GBps per ssd or 8x8= 64 Giga bits per second per ssd. we will be using 24 of these nvme SSDs. iOPS and the speed transfer will be crazy fast for only 25000 accounts running as iMAP/smtp
2
Sérgio Rocha Replied
We are running SM on VM with 10k users in multi-domain in top of a Hyper-v cluster . With all protocols enable but the most used is IMAP.
The cluster its shared with other VMs and it works very well. The most important is to have NVMe, nothing less then NVMe.

The number of user is the less important, really important its the number of message per second, the number of connection, the protocol used and how ative the users are.

SR
0
Bilal Mahmood Replied
In our case the number of users will exceed 50,000 within a year hence the need of nothing short of pciexpress 4 or pciexpress 5 nvme storage!

Reply to Thread