We have about the same number of domains and email accounts and run SmarterMail Enterprise comfortably on a bare-metal server with a single Xeon X3450 2.67Ghz 4 Core/8 Thread CPU and 16GB ECC RAM, Seagate Constellation ES.3 SAS Drives running in RAID-10 on an Intel RAID SAS Controller.
We have run SmarterMail as a VM before but in our circumstance (we have very high volume of traffic) found I/O to be too high and it caused periodic performance issues for any other VMs using the same SAN.
We have also run SmarterMail on a Dual Xeon 2.8Ghz CPUs (4 Cores total) and 1GB ECC RAM, Dell 15K SAS Drives running in RAID-5 on Dell Perc6 Controllers and it was perfectly fine 99% of the time, but 4 times a year (the weeks of Valentines Day, Mothers Day, Black Friday, and Cyber Monday) it couldn't keep up with peak traffic anti-spam checks and the Spool would get to 10K messages with an average of a 40 minute delay in delivery (obviously Memory limited). We eventually added another old Dell 1850 server like this as an Incoming Mail Gateway running SmarterMail Free Edition to resolve the problem (although upgrading the RAM probably would have sufficed).
All in all, you don't need much: recommended 4 CPU Cores or more and greater than 1GB of ECC RAM, and fast enough HDDs (or these days just use SSDs) should be plenty enough for the number of users you have.