We don't have nearly as many domains (about 110), but over the last 2 months we've had to increase memory a few times. SM is installed as a VM so it's easy for us to add memory. For the longest time it had 20Gb allocated with between 6-10Gb in use. Then starting since December we've gotten a few down alerts for the system and check to see that memory is fully consumed. The system is now up to 30Gb with 20-25Gb in use during peak times. We've also been doing regular updates to try and resolve various issues, so yeah it's hard to say which build brought this on.

I have a SmarterMail server with about 150 domains and about 750 users.

No issue with memory or CPU usage

This is my normal CPU and RAM activity (please note that this is a Hyper-V VM and I use Dynamic Memory with max 16GB of RAM):

Thisis the RAM used by processes:

Hello,

Thanks to all for the feedbacks.

It is strange, it seems some users have the same issue (memory usage growth since december) that we have and some other have very low memory footprint on their process.

Process memory usage shows it started changing a bit in december so probably some changes in memory management since a particular build of december.

It's not really a big problem if everything runs smooth as we can easily add ram, but we would like to avoid the process eating all system memory to leave a bit of ram for system cache and other processes.

Thanks again for the feedbacks.

On my end: 50GB in ram in Task Manager (96GB RAM available)

Environment: virtualized in ESXi

Recently I had to move VM from 2x4core CPU to 2x6core CPU because there was situations when SM service and IIS process needed more CPU resources.

When it comes to number of domains and users this is not so easy information to get from SM admin section. Domains section is not showing summarized numbers. 226 pages with 25 results per page which gives about 5650 domains configured in SmarterMail instance.

EDIT: I didn't had any issues with RAM actually (more CPU like I've mentioned) but yes I remember that few months ago SM process was using less RAM but for me this is not a problem (for now at least).

523 Domains

610 Domains

512 Domains

Thanks a lot for your feedbacks

Ours normally uses 40 - 50GB of RAM. Highest I have seen is 56GB.

Just a small update on my end. Today (build 7699) I see 76GB of RAM (SM process, w3wp is using aboug 1.8GB)

Build 7719 Today:

60/128 (47%)
57/128 (44%)
59/128 (46%)
43/128 (33%)
93/192 (49%)
30/64 (47%)

Hello all, thanks again for the feedback.

So I guess SmarterMail is allowed to make use of the memory to keep things in cache and that the memory usage increase is something normal.

At least until it eats all system memory and leave nothing to the system that I hope will not happen.

Kind regards.

I don't mind giving it all the memory it needs as long as the software is stable.

I think SM has a fair memory use. Have you ever tried to use MS Exchange? It eats up every drop of RAM you have! Far worst than SM...

Chris, same. I will toss RAM at SM all day, if it increases performance and stability, SM can eat all the RAM it likes lol.

I think we all agree on this. As long as it it is stable, it can eat whatever memory it wants :)

Our shared SQL Server instance eats nearly a full TB of RAM, it never stalls, crashes or even sneezes, it's a rock. So, SmarterMail, if you want 64GB, 128GB, 512GB of RAM, you have it :). If it helps, we will give it.

I just upgraded Smarter mail from 7699 to 7719 and my CPU & RAM usage has gone insane.

We typically use around 8-11GB Ram and now its using 80GB ram and CPU really high, normally the fix to reduce cpu/ram is to disable greylistings (or delete the greylistings file and restart service) on this occasion that hasn't helped and our SM service crashes several times an hour!

I opened a ticket a few hours ago and awaiting smartertools support to open to hopefully help me.

My server crashed last night the service just stopped, I submitted a ticket. I may need to add more RAM to my server., noticed I am using 92% of 16GB Here are a couple of the interesting events just before the crash.

Log Name:      Application
Source:        .NET Runtime
Date:          2/25/2021 12:34:30 AM
Event ID:      1026
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      XXXXX
Description:
Application: MailService.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.OutOfMemoryException
   at System.Threading.Tasks.Task.AddException(System.Object, Boolean)
   at System.Threading.Tasks.Task`1

This was another

Log Name:      System
Source:        Microsoft-Windows-Resource-Exhaustion-Detector
Date:          2/25/2021 12:27:41 AM
Event ID:      2004
Task Category: Resource Exhaustion Diagnosis Events
Level:         Warning
Keywords:      Events related to exhaustion of system commit limit (virtual memory).
User:          SYSTEM
Computer:      xxxxxx
Description:
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: MailService.exe (5268) consumed 41058426880 bytes, clamd.exe (5780) consumed 1765343232 bytes, and tomcat8.exe (2184) consumed 693596160 bytes.

HI

Do you use greylistings? if so try disabling them and restart SM Service and see if this fixes it for you.

You are going to need more memory Patrick. With latest builds, I'd say at least 64GB. We are running 128GB all the way up to 192GB.

Thanks now I need to be new hardware. I do not make enough money off of SmarterMail to keep this business model going. I currently have one user who keeps having a folder fill up with files. He use to have a 3GB mail file, now it jumped to 30GB. I am at the mercy of the developers with no word all day.

Patrick, you need to move mail services to a new server or limit the amount of resources that other services can use.

Exception Info: System.OutOfMemoryException is typical of of a 32bit process that exceeded the amount of memory available to it of 4GB or .net garbage collector hitting its max

Jade, SmarterMail isnt a 32-bit process, so the 4GB limit doesnt apply here. 16GB of RAM though is a pretty small amount of RAM in today's world.

All,

We do know SmarterMail has been using a bit more memory over the last few months and much of this has to do with the substantial encoding changes we have made to better accommodate emails in all languages, etc. This requires a lot of string manipulation and changes in encoding types, which impacts memory.

In this new upcoming release, we have optimized some of that code while still maintaining all the work we have done. As a result, all customers will see a reduction in memory usage.

With this additional encoding work, it exaggerated some environments where customers had removed message size limits and attachment size limits. In the last month, we have been made aware of customers sending and receiving messages from 750 MB to 1.5 GB. This will put an AWFUL strain on a mail server, which is why Exchange and Office365 limit emails to 500 MB, something we will be enforcing in our next release. SmarterMail's File Storage feature, and using links in messages for files, is a critical tool for swapping large files with people. Whenever possible, this should be used in place of actually attaching large files to messages.

There have been couple memory threads but huge spikes and issues and it was related to these large messages. As soon as we helped the system administrators realize these large messages were being sent and limiting message size limits, all issues were resolved.

We upgraded to 7719 last Monday (a week ago) and the server crashed on Thursday. Getting lots of memory warning emails today, and SM is hovering between 80% and 85%. CPU usage is bouncing from floor to ceiling like a roller coaster several times per minute. We have 24GB RAM on the VM. Looks like I'll have to upgrade that significantly, which is fine, since we host about 420 mailboxes in about 140 domains (which is a guess, since SM doesn't show those numbers on the main domain screen anymore). I'll also start limiting attachment sizes, and maybe turn off Greylisting?

Last November we had a user with a corrupt mailbox file (GRP file?) and it took the server down at least twice a day until we moved the install to a new server (inside a VM), upgraded to 7699 (December version), and instructed the user to remove his account from Outlook and re-add it. I know that's not what's going on here, but for future researchers, corrupted mailboxes can play havoc on SM CPU and memory usage.

@echodreams

Im well aware that SM is not a 32bit process, my suggestion to Patrick still stands - running a web server and mail server on the same server with limited resources is not a good idea.

If possible, segregate those into individual virtual servers or if budget allows physical servers.

@Patrick,

We were in the same boat a few years ago, re: not making money off hosting email. We changed our model by no longer giving out a free email account with a website, and increasing our fees by a buck/mo. This is a service-add for us, not wanting to send all our clients to Google or Microsoft. Why should they get that money? Now we earn enough to make it worth our while. Big help was investing in enough metal and continuously upgrading to the latest release to ensure things run smooth enough that I don't spend long hours dealing. Yes, there is a bumpy road after some upgrades, but not all. Bottom line, it's worth its weight to have a good email server at my disposal for setting up customer email accounts for me and family, and my customers, who are all very happy not to have to deal with global conglomerates. Stick with it.

I think you can run a web server and a mail server side-by-side. We host quite a few customers who run SmarterMail alongside IIS (since IIS is already running with SM), they run their website along with SM on a 32GB server with webmail, their main site (maybe a few others) without problems. Most of these users only have a few hundred if that SM users.

I wouldnt do it in our case, were we have 10s of thousands of users and need every drop of resources dedicated to SM.

I just moved our SM instance to a new Windows 2019 server 2 weeks ago. Memory usage was fine up until yesterday when total system usage started hovering around 90%. Well this afternoon SM memory usage hit 96% and system CPU usage dropped to nothing. Needless to say webmail, etc was all unresponsive and I had to kill the SM service and restart it. 

With the previous server I had resorted to scheduling weekly SM service restarts to avoid the periodic memory issues. I had hoped with a fresh server that would no longer be needed, but I find myself scheduling service restarts yet again. 

We're still on 7776 b/c I was trying to keep that stable while working on the new server. Has anyone here that's had memory issues in the past using one of the newer builds? Is memory usage more stable?

The latest builds don't have the consume-all memory issue. Currently ours are hovering around 64 out of 128, about 50% usage.

We're running v.7776 on a Windows 2016 VM with 24GB RAM and two hard drives.

I do not have an installation on your scale, but I do have intermittent problems with SmarterMail service consuming almost all available memory and running very slowly as a result.   Restarting the service has solved the problem.   After reboot, memory does not return to its previous levels.  Since the problme is intermittent and infrequent, I do not think that an underconfigured server is the primary problem.   The problem has appeared across several releases.   After the last time this happened, I configured a scheduled task to reboot SmarterMail weekly, and the problem has not reoccurred yet.

Hi all, 

You certainly shouldn't have to restart the SmarterMail service regularly in order to avoid CPU/Memory spikes, and we'll want to look into this more so you don't have to. If you're having intermittent or regular CPU/Memory spikes, we'd ask that you please submit a ticket to the Support Department. The support technicians can offer feedback on how best to capture these spikes and get the information to the development team for further review. Specifically, we'll need to get a DotMemory and DotTrace while the resources are spiked so the development team can identify any areas of concern that may be using up those resources unnecessarily. 

Thank you,

Past week or so we've noticed that the service will use 50% of RAM for a couple hours then go back down.

We're running v.7729 on a Windows 2019 with 128GB RAM  C: SSD , D: SAN Storage.

But Memory seems to go up by 20% per day.

Why does memory keep accumulating?
Is there something waiting like a zombie while doing a service?
I can't understand.

 

Might want to check for corrupted mailbox files.

It's an older thread but I concur with what Mik Muller says. We experienced the same issue with progressively increasing high CPU & memory usage spikes over time in SmarterMail until the service eventually crashed due to a generic .NET cld.exe error. During a recent migration to another physical server we discovered five .grp files amongst five different users (out of a couple thousand) that were corrupted and > 8GB in size. After removing those corrupted grp files SmarterMail has been steady at 3% avg CPU (up to brief 20% spikes during peak usage) and a steady 16% Memory usage (20 of 128GB). Disclaimer: we do anti-spam checks on Incoming Gateway Servers, so we have a much lower CPU & Memory usage for our user base.

A quick search of your \SmarterMail\Domains directory structure for "*.grp size:Gigantic" and "*.grp size:Huge" can pay dividends.

Also, check your concurrent IMAP connections. We noticed that older versions of MacOS X (I believe it was High Sierra) were resulting in IMAP connections from a single client getting into the 200+ connections which was also causing high CPU & Memory usage prior to eventual crashes (one client isn't an issue but when you have dozens with the same problem it eventually adds up as it seems those connections weren't timing out and were just multiplying). Contacting those clients and having them upgrade to a more recent version of MacOS X resolved the high concurrent IMAP connections coming from their Mac Mail.

Hello Everyone, 

I just wanted to reach out and request that you guys try the version of SmarterMail that we just released. 

This version has lots of performance improvements and better error handling. 

It should be a substantial improvement over the previous versions. 

I hope you all have a safe and fun holiday.

Kind Regards, 

Everyone,

Please upgrade to Build 8025, the latest build that we have been emailing and communicating about. There were a number of scenarios where broken email clients (primarily Outlook clients) could attack a server with hundreds of thousands of bad commands per day. This resulted in higher than expected CPU and increased handles / memory which would impact the Garbage Collection process in the .NET Framework and cause a variety of unexpected issues that were very difficult to correlate. 

Even though we manage bad clients much better now, you still don't want your server getting hit hundreds of thousands of times a day with bad commands. Unfortunately there is nothing we can send to these clients to stop them because, in most cases, they are in loops for various reasons out of our control. So, we have added potentially bad clients into the Administrative logs so you can contact these users to fix them. As we learn more about different ways clients can get totally out of whack and impact your servers, we will add them to this log as well. Right now, we have logging for misconfigured Outlook MAPI clients, and you can find instructions on how to find and resolve those clients here: https://portal.smartertools.com/kb/a3559/find-and-resolve-misconfigured-email-clients.aspx#

In any case, we evaluated all aspects of SmarterMail with a wide variety of customers and servers over the last couple months to really narrow down all the unique and random things we saw customers complain about sporadically. They were torture to find, but we fortunately we had a lot of success so it was rewarding as well. :) And we're really happy with Build 8025 as a result. 

Healthy and Happy Holidays to Everyone!