We are experiencing a memory leak in a Windows SM 9229 after upgrading from 9182. Downgrading does not fix the issue.

Problem reported by Gabriele Maoret - SERSIS - 4/17/2025 at 5:12 AM

Resolved

We are experiencing a memory leak on a SM 9229 installed in a dedicated Windows 2019 Server operating system after upgrading from 9182(where everything was working fine...).

This server has only 62 users...

After upgrading, the SmarterMail service started to increase the RAM used little by little, until it completely filled the server's RAM (12GB) within 1-2 hours.

Before that, the usage was constant between 3 and 4 GB.

Unfortunately, going back to 9182, strangely, did not solve the problem, so we upgraded again to 9229.

We are now restarting the service every hour to mitigate the issue, but this is a really big problem...

EDIT: I corrected the version I started from because I had mistakenly written 9168, while the correct one is 9182

Gabriele Maoret - Head of SysAdmins and CISO at SERSIS

Currently manages 6 SmarterMail installations (1 in the cloud for SERSIS which provides services to a few hundred third-party email domains + 5 on-premise for customers who prefer to have their mail server in-house)

54 Replies

Reply to Thread

Richard Laliberte Replied

4/17/2025 at 5:40 AM

I would be curious to know if the problem persists on a fresh install vs upgrade / downgrade. Are you able to do an export of all the data, then completely un-install and re-install 9229 (or 9168 which ever you prefer)? There have been a few memory leak issues reported by various people on various platforms and builds, so i'm more curious if something in the update process changes the actual accounts causing the memory leak vs SM introducing a memory leak?

We are currently running on 9182 on a dedicated windows 2019 server and haven't noticed any issues. We are holding off on the upgrade because of this specific issue.

Gabriele Maoret - SERSIS Replied

4/17/2025 at 6:20 AM

Sorry, my mistake... the previous version was 9182 for me too (I've just edited the post to correct). I had no problem with this version either.

@Richard Laliberte : unfortunately I can't do this easily... This is an on-premise production server in a customer's data center.

Since I can't create a new server in the customer's data center, I would have to copy all the data to MY data center and set up a test server there.

But then I would be missing 62 users who can simulate the customer's mail usage...

Gabriele Maoret - Head of SysAdmins and CISO at SERSIS Currently manages 6 SmarterMail installations (1 in the cloud for SERSIS which provides services to a few hundred third-party email domains + 5 on-premise for customers who prefer to have their mail server in-house)

Richard Laliberte Replied

4/17/2025 at 7:03 AM

a bummer. That will make it a bit more difficult to track down the actual cause of the leak. Guess we will hold off as well upgrading so we don't run into similar issues... although i thought i read somewhere that someone got a 9230 build and it was working well.

Curious though, 1) is the install itself a long running upgrade, or did you do fresh installs at some point along the line. and 2) are the user accounts also older, maybe going back several years worth of updates? (programmer in me coming out now trying to help SM get all the data they can lol) We are looking at upgrading our windows box at some point from 2019 to 2022 (possible 2025) and i've been on the fence about asking for a fresh install and running an import wizard on SM just in case there is left over junk code (we've all seen it with anything Microsoft related lol)

Gabriele Maoret - SERSIS Replied

4/17/2025 at 7:42 AM

In the meantime it seems to have calmed down a bit and now it doesn't happen so often... Who knows!

To be safe, I set up a PowerShell script that checks the RAM used by the "MailService" process, restarting it if it exceeds a certain number of Megabytes...

I run it at regular intervals every 10 minutes, so that if the RAM used exceeds a certain level it restarts, otherwise not (this way I avoid unnecessary restarts...)

It works wonders!

Gabriele Maoret - SERSIS Replied

4/20/2025 at 4:30 AM

This seems to be fixed in 9238.

Now it's two days without memory leaks

Gabriele Maoret - SERSIS Replied

4/22/2025 at 7:14 AM

Never mind... The reboot for too much RAM usage was triggered this morning...

So MY (I don't know if others match it...) problem remains even with the latest 9238 (Windows)

Richard Laliberte Replied

4/22/2025 at 8:00 AM

I hate to say it, but i think this is going to be a lot harder to track down than we hoped. The fact that it's not affecting everyone, and the problem persists even when downgrading, makes this a needle in a hay stack problem.

Curious though, have you checked the web logs around the time leading up to the last reboot? We do web development across a number of servers, and we have noticed a big up-tick recently in attacks. Long shot but something to consider if SM changed how certain processes are handled. A burst to certain ports or even login screens with a mis-configuration in logging or something COULD cause a memory issue...

Tim Uzzanti Replied

4/22/2025 at 3:30 PM

Employee Post

Gabriele,

The reason your issue came back is because we provided you a custom build which modified settings for .NET 9 (wasn’t a code change), which encourages Garbage Collection more frequently because your server was not feeling memory pressure to do so and / or because your CPU was occupied in a way that told .NET not to prioritize Garbage Collection.

We wanted to see if that would help .NET 9 perform better on your server / configuration which it seems it did. We also sent that Custom Build to a couple of other customers and they had similar experiences. With a little more QC internally, we should be including those settings changes to .NET in our next release.

Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com

Gabriele Maoret - SERSIS Replied

4/22/2025 at 10:49 PM

Thx for the info, Tim!

I'll wait for the next release with this fix included

Gabriele Maoret - SERSIS Replied

4/24/2025 at 8:08 AM

Hi Tim!

Do you think this new patched release will be released soon this week?

I ask because in these two days the situation is getting worse, with a blocking problem of too much used RAM that forces to restart the SmarterMail service very often (sometimes 20 minutes, sometimes a little more than 1 hour...)

Derek Curtis Replied

4/24/2025 at 8:36 AM

Employee Post

Yes, this week's release will have the .NET settings changes we made that were part of the custom build we sent out last week.

Derek Curtis COO SmarterTools Inc. www.smartertools.com

Cris Mead Replied

4/25/2025 at 4:35 AM

Derek, can you confirm Build 9245 has this fix?

guessing it's this line: Efficiency: Added settings to .NET to improve memory management.

Richard Laliberte Replied

4/25/2025 at 5:37 AM

Just out of curiosity, but is there any way to manually adjust Garbage collection? i know this new update is supposed to improve the settings, but it would be nice to be able to adjust it base on environment or run trigger it manually if things get out of control.

Gabriele Maoret - SERSIS Replied

4/25/2025 at 6:57 AM

9245 installed.

Let's see if it solves this issue...

Cris Mead Replied

4/25/2025 at 7:56 AM

We are having this issue too, We can't see this affecting the system counters, save maybe in page faults - where smartermail has 600+millions (upwards of 10,000 hard PF/sec). How are you monitoring .NET for these memory issues?

This might be the issue that drives us to another email service, I'm under a lot of pressure from decision makers to "get this fixed" or consider moving to another mail provider.

Gabriele Maoret - SERSIS Replied

4/25/2025 at 4:18 PM

In this few hours 9245 seems to behave better with RAM usage, but it's too soon to be sure...
I'll report again in 5-6 days to tell you if my issue really is fixed

Gabriele Maoret - SERSIS Replied

4/28/2025 at 3:43 AM

Unfortunately this problem persists with the new version 9245...

Previously the Windows server had 8 GB of RAM and never had any problems.

Starting from the update to version 9229 (and then the subsequent ones...), the SmarterMail Service ("MailService") consumes RAM to the point of completely blocking the server.

I updated the server's RAM to 12 GB, but it still crashes with memory leaks on the SmarterMail service.

Note that this is a SMALL server with only 60 mailbox...

Fortunately, my workaround works (stop service, 10 seconds pause, restart service when SmarterMail Service exceeds 6 GB of RAM used), but this, during a workday, means restarting the service very often, at least 8-10 times...

Brian Bjerring-Jensen Replied

4/28/2025 at 4:16 AM

How many CPU cores does the server have? Same amount of users but no issues.

Gabriele Maoret - SERSIS Replied

4/28/2025 at 4:45 AM

Hi Brian! I have 6 vCores.

I had never had any problems until v.9182 either.

Then when I updated to 9229 (nothing else was done, just the SmarterMail upgrade...) these problems immediately started.

Unfortunately, the downgrade to 9182 does not solve it, and neither do subsequent versions...

I also increased the vRAM from 8GB (which were always enough BEFORE) to 12GB, but I did not solve it...

Fortunately, the script that checks the RAM usage and if necessary the automatic restart of the service (scheduled every 15 minutes...) makes the problem transparent (or almost...) to users, but it still remains.

...I suspect that it is a problem specific to this installation, I will be forced to open a ticket...

Brian Bjerring-Jensen Replied

4/28/2025 at 5:31 AM

How is your memory looking when using performance monitor?? Is it the standbymemory thats increasing?

Gabriele Maoret - SERSIS Replied

4/28/2025 at 8:07 AM

Good question, but I think the issue is not in StandBy Memory (maybe I'm wrong..)...

This is the situation NOW, with few users (12) at work:

Thsi is a OK situation. During the working hours with 30-40 user working, the issue is that sometime MaiService.exe grows to 6-7-8-9-... GB an the the server became dull...

If I restart the sevice, it stay at 1-3 GB for some time and then start growing (this only during working hours. from 5pm to 8am no issues)...

J. LaDow Replied

4/28/2025 at 8:50 AM

Some standby cache usage is good, and quite beneficial to SmarterMail. Beneficial on SM and MailEnable (when we ran it). @Brian - if you have extreme standby-cache usage, one thing to do is see what application is loading down the cache. We've seen most problems with that on user systems and less on servers.

SM's main process being the one going overboard in memory usage is a completely different issue than Windows Standby/Prefetch cache system.

MailEnable survivor / convert --

Derek Curtis Replied

4/28/2025 at 9:01 AM

Employee Post

Gabriele, I'll have someone reach out to you about this as your previous ticket was closed. I can confirm that the .NET settings changes that were in the custom build ARE in last week's release. So we'll need to do some more troubleshooting with you on this.

Derek Curtis COO SmarterTools Inc. www.smartertools.com

Brian Bjerring-Jensen Replied

4/28/2025 at 9:14 AM

Issue is that I have seen standby memory causing issues on SM before and I found the ,exe file that kills standby memory cache and that has made the server pretty stable over many releases.

If its too high, then the server halts and everything is painfully slow.

J. LaDow Replied

4/28/2025 at 10:26 AM

Agreed when the standby cache gets out of control there can be issues.

We have noticed the main driver of bloat on our servers is MS Defender and SM users with large "inbox" folders - MailEnable also had this issue. I just checked our server, and our most active SM stuff in standby cache is a couple users we have who are currently online in webmail or IMAP. We don't use EWS or MAPI. The index files SM maintains are also in our standby list.

During the week, we average 100 users online at a time with about 150 active connections - either webmail or IMAP - before I flushed it, we had 20gb in use in standby and 4gb in use on SmarterMail processes on a 4 vCore server. {edit} The only time it reboots is for updates and we're on an older SM install. {/edit} We've never seen signs of cache pressure yet, as far as behavior/performance of the server goes. It was WAY worse with MailEnable.

{edit/added}

FWIW - you've piqued my curiosity and now I'm going to spend some time monitoring our server's behavior a little more closely. I've dealt with problematic standby cache issues to the point where I wrote a windows service that background polls and flushes the cache based on a couple registry settings. I wrote a desktop version but never really "finished" it. I'll trace a few things for a while, then install the service and continue monitoring. We've got Grafana and Prometheus running on this server so it'll be easy to gather data.

The thing about standby cache flushing too often is it's self-defeating in many instances as it increases disk read traffic - especially in situations where lots of files are actually in use - you get surge of reads/backlog on throughput and make the situation worse in some cases.

MailEnable survivor / convert --

Brian Bjerring-Jensen Replied

4/28/2025 at 11:08 AM

Unless you run it bare metal, the cache is using disk as well as storage since virtual memory is using the same storage as files/disk.

So it doesnt matter for performance. We run All flash Arrays and havent had performance issues yet.

Gabriele Maoret - SERSIS Replied

4/29/2025 at 12:15 AM

This is an example on how MailService.EXE has grown in the latest 27 minutes:

15 seconds after process restart:

3 minutes after process restart:

After 11 minutes:

after 14:

25 minutes:

30 minutes:

After 35 minutes:

41 minutes:

48 minutes:

55 minutes

Gabriele Maoret - SERSIS Replied

4/29/2025 at 3:00 AM

Incredibly, after the last restart of the service at about 10:05 am today (it had exceeded 7GB of usage, which is the limit I have set now on my restart script...), the service seems to be stable on 2-3 GB of RAM usage...

I'm monitoring with PerfMon, with a sample every 12 seconds (the graph contains 3 hours of data maximum...), let's see how it goes ...

Gabriele Maoret - SERSIS Replied

4/29/2025 at 4:53 AM

It continues to be stable between 2 and 3 GB...

I'm monitoring...

Richard Laliberte Replied

4/29/2025 at 5:36 AM

i don't suppose there are any accounts that you noticed that haven't connected since the reboot? Maybe you have a corrupted user account?

Gabriele Maoret - SERSIS Replied

4/29/2025 at 6:42 AM

@Richard : I have a suspicion about an account, I'm waiting to see if it connects and changes anything...

I have configured the "suspicious" account on my PC for now to see if it increases RAM usage... for now it's all ok:

Richard Laliberte Replied

4/29/2025 at 8:06 AM

hope this is the case. this memory leak issue is honestly the last item we are watching before we do our next update lol

Brian Bjerring-Jensen Replied

4/29/2025 at 8:08 AM

Has the server been updated recently?

Gabriele Maoret - SERSIS Replied

4/29/2025 at 8:33 AM

@Brian : no updates to the server other than SmarterMail to the new releases.

@Richard : connecting the "suspicious user" on MS Outlook MAPI in my PC does not trigger the issue... So I'll monitor the server for the next days (no issues till now after the latest restart at 10am):

Gabriele Maoret - SERSIS Replied

4/29/2025 at 11:30 PM

Today the issue restarted, so I'll go on with ticket and RSAA (in the graph below you can see the memory leak and my script that restarts the MailService ...):

Gabriele Maoret - SERSIS Replied

4/30/2025 at 5:29 AM

Brian Bjerring-Jensen Replied

4/30/2025 at 5:34 AM

Have you tried to turn antivirus and all other 3rd party apps runnning off?? To isolate the mailservice.exe running to see if thats really the culprit?

Gabriele Maoret - SERSIS Replied

4/30/2025 at 8:52 AM

It seems to be related to a single user... If that user is logged out of MS Outlook MAPI, everything works fine, if that user logs in to Outlook (MAPI), RAM usage starts to increase...

Now I need to find out if it is due to corrupt data in his account on the server or if it is his PC that is having problems...

Jack. Replied

4/30/2025 at 9:15 AM

@Gabriele, I think the problem is bigger now; finding an email account that affected the entire service and the server is a very, very complex task.

It's important that SmarterMail has a tool that identifies email accounts that may have data corruption.

Thank you for the details of your case, they are helpful to us as well.

J. LaDow Replied

4/30/2025 at 11:00 AM

There definitely should be some kind of integrity checker for the configuration files - and something for the command line that will verify the condition of the message data files as well.

It's understood that it's on the server side - but this level of diagnostic would be useful.

Does SmarterMail expose any Performance Counters or other endpoints that can be monitored? At some point, Grafana/Prometheus could be integrated to monitor health as clients connect/disconnect, etc. I would contribute to something Marketplace related if the endpoints were available.

MailEnable survivor / convert --

Brian Bjerring-Jensen Replied

4/30/2025 at 11:15 AM

Its mindblowing that one single user account can take down a server over time.... Imagine the impact it could have on an ESP.....

J. LaDow Replied

4/30/2025 at 6:02 PM

We used to have this exact same problem with MailEnable - but haven't had it yet on SmarterMail (but we've held back upgrades since 2023 at this point.

MailEnable survivor / convert --

Brian Bjerring-Jensen Replied

5/1/2025 at 12:04 AM

Have you tried the good old 8930 build??

J. LaDow Replied

5/1/2025 at 4:45 AM

We quit paying maintenance and support because for over a year there were no stable updates. In the last two+ years, there's been three "notably stable" versions - 8930 was one of them but we had already given up at that point. We're a small shop who can't afford to lose customers over "bad updates" and can't afford to give up the man hours to do the testing and diagnosis on a product we paid for as enterprise grade. So we're still waiting.

MailEnable survivor / convert --

Tim Uzzanti Replied

5/1/2025 at 11:26 AM

Employee Post

J. LaDow,

You’ve got to be kidding. We support over 20 million users on SmarterMail, including servers with tens of thousands of users. SmarterMail is incredibly stable, and we typically address bugs within weeks of discovery. We also provide custom builds for testing. You won’t find another company more involved or responsive to its customers—we don’t have a single vendor that operates at the level we do.

Gabriel’s customer has a calendar file exceeding 100MB, all of which must be loaded into memory which blows up substantially in memory because of recurring appointments and associated ical data. This is a key factor in the memory usage being reported on his server and at some point, how a customer uses our product plays a significant role in its behavior.

We’ve been responding to Gabriel within hours, while his replies have taken days, slowing progress. Gabriel just provided us with access to the server and the credential didn't work, delaying the progress.

I get frustrated when people jump on the bandwagon without understanding the full context. It’s counterproductive and only gets in the way of finding solutions that work for everyone. So don't do it, we will remove you from the community.

Tim Uzzanti CEO SmarterTools Inc. www.smartertools.com

Gabriele Maoret - SERSIS Replied

5/1/2025 at 4:44 PM

I agree with Tim, SmarterTools crew are amazing at help us...

Unfortunately, I'm busy on other tasks and today is Holiday here in Italy (and the difference in time zone don't help...), so I can't be very fast and responsive as I would, but they definitely support us very well

Sabatino Replied

5/1/2025 at 11:38 PM

A year ago I noticed that in the error file I had some reports related to the calendar

[2024.03.14] 11:22:53.129 [FileFunctions] File is over 15mb, this might cause reduced performance. Size: 23.9mb Filepath: f:\SmarterMail\Domains\xxxx.it\Users\daniele\folder-60003.json

which was just a calendar error with recurring appointments

@Gabriele did you have anything in the log file?

Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy

Gabriele Maoret - SERSIS Replied

5/2/2025 at 1:13 AM

At the moment I have the LOGs set to EXCEPTIONS ONLY and I don't see anything that helps me with my problem (at least in the EWS and/or MAPI logs...), but I have an open ticket and SmarterTools are analyzing the situation...

I also reported the "suspicious" user, so they can check if he has something strange.

Sabatino, in which LOG did you see this report?

Did you also see it with EXCEPTIONS ONLY or did you have to raise the logging level?

EDIT: I found it: it's in the "ERRORS" logs.

I see this errors (many of them...) for the very user who is my #1 suspect:

<<<<<<

[2025.04.30] 11:31:11.478 [FileFunctions] File is over 15mb, this might cause reduced performance. Size: 184,8mb Filepath: S:/SmarterMail/Domains/DomainXXXX.com/Users/UserXXXX/AttachedFiles/attachedfiles.json

>>>>>>

Do you think this could be the culprit?

And if so, how can I fix it, in your opinion?

Sabatino Replied

5/2/2025 at 8:36 AM

I don't know @Gabriele.

But when you highlighted the problem of the calendar size I remembered this thing.

At the time they suggested autoclean but I didn't set it.

I corrected that user who had set a daily recurrence on the calendar by increasing the size.

In any case I think that edge cases like this should be managed server-side. A user cannot cause the consumption of all the server memory.

Also, as said more in the past, I hope that SM improves the log system because in my opinion it makes our life difficult. Maybe now with the marketplace maybe we can add some custom modules (I don't know, I haven't looked into it yet)

Errors loading a user and/or a domain should be reported more clearly.

Even in the past I had an error loading a json of a domain that wasn't recorded in the log.

I reported it via ticket but I don't know if it was ever solved

Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy

Richard Laliberte Replied

5/2/2025 at 9:08 AM

If this is in fact the case, and it is a problem specifically with Calendar size, i would hope that SM can implement some type of archiving system or something that would only store say current month and up-coming in memory, but i'm guessing this is a limitation because of things like MAPI that need full access.. maybe even a more direct warning, instead of log files send an email (similar to when the mail box is almost full) stating that, in this case, a users calendar is becoming to large, please clean or archive?

Gabriele Maoret - SERSIS Replied

5/2/2025 at 12:52 PM

I really hope that the investigation and subsequent solution (as soon as it becomes clear what this solution is...) to this problem of mine will inspire the SmarterTools team to do 2 things:

1 - improve SmarterMail's resources and exception management so that it can isolate any users with similar problems and prevent them from crashing the entire server

2 - give us a better way to identify users with problems, because I imagine how difficult it would be to find a single user causing problems on a server that has thousands of regular users... In this case, with only 62 users it was "easy", but in the case of thousands...

Gabriele Maoret - SERSIS Replied

5/3/2025 at 1:17 AM

I suspect that the problem is due to the "AttachedFiles" folder of the offending user...

This folder contains hundreds of thousands of files (while all other users have only a few dozens, rarely more than a hundred...). See image below.

Could this problem also be connected to calendars, for example because these "AttachedFiles" are files attached to a recurring event?

Gabriele Maoret - SERSIS Replied

5/6/2025 at 12:28 AM

Marked As Resolution

MY PROBLEM SEEMS TO BE SOLVED.

For your information, I report here the results of the analysis and the solution we arrived at.

As per previous posts, I confirm that the problem was due to a single user with an anomalous number (hundreds of thousands...) of files in the "AttachedFiles" folder, due to endless recurring appointments that contained an iCAL (.ICS) attachment.

These "anomalous" appointments were generated by an invitation to meetings created with Google Calendar and Google Meet by an external user (the external user is the Owner of my client company, which uses Google because he also owns 20 other companies that all use different systems and therefore he is fine with it. Oh well...).

Of course you understand that I CANNOT delete the appointments...

The final solution was to do this:

1 - export the offending appointments in .ICS files via the Webmail menu. This way everything is saved EXCEPT the problematic attachment.

2 - delete the appointments on the webmail (the whole series). This way the "AttachedFiles" file repository is cleaned

3 - re-import the appointments from the .ICS files previously saved in point 1. This correctly recreated all the recurring appointments with all the details WITHOUT the iCAL attachments that were weighing everything down.

NOTE: to find any users who have these problems you can search the "error" log for reports similar to this one:

<<<<<<<<<

[FileFunctions] File is over 15mb, this might cause reduced performance. Size: 184.8mb Filepath: S:/SmarterMail/Domains/DomainXXXX.com/Users/UserXXXX/AttachedFiles/attachedfiles.json

>>>>>>>>>

In the ticket I opened I was assisted by Ray Burd, who confirmed that in the past (v. 9084 approximately) there was a bug that could create these problems with attachments from invitations received from Google, but he says that in the latest versions it should have been fixed.

The problem was "venial" if you received an invitation for a single event, but if the event is infinitely recurring, then it causes the RAM leaking that I saw. Let's hope it is truly solved...

Of course I asked him to do further checks on the correction of this BUG!

P.S.: Thanks to Ray and Tony for the help!

Oliver Replied

5/6/2025 at 7:58 AM

@Gabriele thank you for the detailed report

Back to Community Threads

Please leave this box unchecked

Reply to Thread

Enter the verification text