Huge memory in SM 15.3.6095
Problem reported by Neal Culiner - September 17, 2016 at 7:00 AM
Submitted
I noticed mail delivery was a bit slow from the time I sent a test message until receipt, just not the normal.  I figured I'd take a peek at my Win 2012 R2 box and see what the memory looks like, it's huge.  I've never seen the memory for the MailService this high.
 
Anyone else seeing very high memory on the service?  Sure would be nice to be able to upload an image!
 
This image will be removed at some point as I put it in my dropbox public folder:
 

32 Replies

Reply to Thread
1
Mike Ingram Replied
Mine is not running that high, mine is running 3-4 where yours is 9.5 but I am having tons of issues trying to just import a couple of mailbox with imap.. I just upgraded to the newest version and did not even think about the upgrade and version I was on as being part of the issues. I am going to downgrade after trying to get things to sync for the past day and then see where the memory is at and if I can sync imap box again.... I just posted about having sync issues... and I am on that version as well.... maybe not related but I am about to downgrade to test.. I will let you know when I downgrade in the next few hours what happens with memory...
0
Neal Culiner Replied
I thought there was a place to access prior versions. Do you know where to get the prior version as I want to revert as well.
0
Mike Ingram Replied
I can not find previous version either. Last night I unistalled, rebooted and then reinstalled the current version just to see. I am still having sync issues with the server it seems but the memory used is under 1 now. Not sure if that was because of a reboot of the server, maybe a memory leak in the code or the server was just busy.. I am trying to migrate mail but not having much luck. I am thinking about going back to the last version of 15.2 since that is the only one I can find on my server downloads (lesson learned to keep my old downloads). I can NOT find any download links to the old version on the website.. they make it very hard to roll back and re-download if needed since we can not find these things. Not to mention most work like this is done after hours and on weekends. Wish smartertools made it easier to find things like this. Before upgrading to the latest version I had no issues with syncing on this server. Now I am not sure if that is the reason or because the boxes I am trying to sync are over10 GB or so, but it just does not go anywhere.. :( The server to server migration said it was done but only pulled partial emails (3-4gb and then the contacts, calendar etc and said it was done.. weird) Frustrated

0
Mike Ingram Replied
I have been watching my memory after I completely uninstalled, rebooted and re-installed on the server and I can say that after doing that even with heavy loads the memory for smartermail does not seem to go over 1 any more. When I did the upgrade I did not uninstall, reboot then reinstall.. not sure if that made any difference but just giving you my experience. Two of the sync's I had going just finished as well.. only two week periods but better than before..... I am going to leave mine at the newest install and see what happens. again, my memory is staying way down after the uninstall, reboot and re-install...
0
Neal Culiner Replied
I followed your suggestion of uninstall, reboot, reinstall and on Win Server 2012 R2 (64-bit) I am already over 3 GB and climbing.
0
Mike Ingram Replied
hmmm, maybe a memory leak ? mine has been pretty stable at 500mb - 800mb with load... I am also on 2012r2... if you want to just try an older version, I can send you a download link to the latest 15.2 version... I know there have been two 15.3 versions but as you notice as well I cannot find them on smartertools website either.. let me know if you want and I can at least get you Version 15.2.6039 (Jul 14, 2016)

When I saw your post, my memory was at 3-4GB which I thought was very high as well. I am really going to watch mine now...
0
Neal Culiner Replied
It's a severe memory leak, 9 GB is ludicrous for an email server. I'll wait to hear from ST in my support ticket. I'll let you know if I need the link. Thanks.
0
Webio Replied
No one is providing any details about server configuration. On my end it is 5.5GB and this is normal memory usage after 52 days of uptime. 4.8k domains, 36k of users.
0
Neal Culiner Replied
My server (ENT edition) is running about 6 domains and maybe 50 total users. EAS and EWS. ClamAV and Real Time AV and after 10 minutes I'm at 3 GB. My screen shot is near 9 GB. That to me, as a programmer, seems like a huge memory leak and excessive memory. I don't recall seeing it in the past and the only reason I looked was slow mail delivery both outbound and inbound.
0
Mike Ingram Replied
Neal, did you talk with ST today? Any chance you got the download link to the last version of smartermail? I am still having sync issues and want to roll back one level just to see. I did not have time to try and track them down today but hopefully tomorrow if nobody responds...
0
Neal Culiner Replied
I did and decided to just accept it is as it is, I don't want to deal with filling out forms, blah blah. Either they see the high memory usage or they don't on their end. I don't have any other issues, it's working fine for me both EAS and EWS with known bugs I reported in the past. If you need a prior version just ask, they've given me links before when asked but it usually didn't help the situation. Hopefully at least my ticket brought the memory issue to their attention. A small server setup like mind should not normalize at 3.5 GB of memory use but I have tons of RAM so it's not a concern, just a symptom.
0
Neal Culiner Replied
I restarted the service this morning, very low server activity, and I'm immediately at 4 GB memory on the service.
1
Neal Culiner Replied
I'd love to know what others are seeing for service memory.  Please post a reply with the memory your SmarterMail service is stabilized at.
0
Matthew Leyda Replied
Windows 2008R2 64Bit
24GB Ram
Dual L5420 (2.5GHz Quad Core)
SmarterMail Version - 15.2.6039
Microsoft.Net Framework Version: 4.0.30319.34209
 
MailService.exe 847Kb
Clamd.exe 462Kb
Kendra Support
http://www.kendra.com
support@kendra.com
425-397-7911
Junk Email filtered ISP
0
Neal Culiner Replied
Also note if you're using EAS and/or EWS as I am.
0
Paul Blank Replied
Wondering here if SM can be designed to use more RAM and/or CPU when desired. Would be a nice feature.
0
Rod Lasky Replied
Employee Post
Hi Neal.  Was this after a recent upgrade?  If so, did you uninstall the previous version first, or install 15.3 on top of an existing version?
Rod Lasky
Technical Support Specialist
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Neal Culiner Replied
I uninstalled, rebooted, installed new version in troubleshooting and the problem remains. Restarted the service this morning and it went immediately to 4.5 GB.
1
keith dovale Replied
Hi,
The last week we have had our 2 server just die on us, today was the only day I was able to catch the server and find out the memory usage was over 8GB of ram, killing the mailservice and the mail server recovered 8gb of ram back, I have tried disabling various options to no avail. This is happening mainly on 15.3, but it is also occuring on 12.5 servers, I can recal that the asp.net 4.6 updates and patches were loaded last week, is this not maybe where the issues have come from ?
0
keith dovale Replied
Ok here is a bit of more info regarding this issue, I began to try and identify a possible cause to this issue, we run declude, which stores its held spam emails in the spool folder under a spam directory, after searching for possible posts, one of the SM guys was indicating that quarantined mails in the spool queue could potentially slow a server dowm, so I moved the declude folder out into the smartermail folder, and i deleted all quarantined emails. and moved all mails out the spool folder.

The mailservice memory usage droped down to 300Mb, I slowly added mails that I removed from the spool, and started with small mails, 1k - 1M files, memory usage grew to around 600Mb and returned back to normal, I then began dropping 1Mb to 4Mb mails into the queue, the memory usage spiked up to around 800mb when I bombed around 20 of these mails in and again the memory dropped back down to 300Mb, I got to mails of 7Mb and dropped 4 of these into the spool, BOOM 2.5Gb of memory usage. The memory dropped back down to stop at 500Mb and has stayed there. The processor usage goes to about 40% and hangs around there constantly.

I then began testing as a client had dropped around 50 of these mails out to clients, 1 mail of 7mb = 675Mb usage - 170Mb usage to process a single file ??? 5 Files spiked up to 2.2Gb usage, 13 files = 3.3Gb usage peaking to 4Gb, the weird part was after the spool delay, and in the queue, but not being delivered yet, the usage stayed at around 3Gb, and as they went into the waiting to deliver queue the memory dropped back down to 500Mb.

I then tried a 20Mb file usage peaked up to 900Mb, 3x 22Mb files peaked at 2.6Gb, 5x 22Mb peaked at just over 4Gb.

I then decided to try looking elsewhere, and took a look at our outgoing spam filtering, I turned off custom rules filtering and remote spam assassin and BINGO after dropping a few 38Gb files in the queue the memory stayed at 450Mb. I re-enabled custom rules and the problem was back, disabled it enabled remote spam assassin and the memory was low. For good measure I tried a few times and it is definately the issue.

We have a few custom rules we run, but we receive a fair amount of mail per day, around 22k mails a day per server, but on incoming spam filtering the custom rules do nothing like this, the memory stays low.

So it looks like custom rules is causing issues on outgoing smtp filtering, I checked this on both my 15.3 servers and one 12.5 server same issue and same solution.

I now am busy looking at the rules to see if there is a specific rule that causes this issue, I will report back later.

0
keith dovale Replied
Ok some more feedback on this issue, I began to drop out custom rules one by one whilst testing dropping large mails into the queues, right down to removing all the rules, and the high memory usage persists even without any rules, disabling the option to run custom rules, drops the memory usage right down, so it seems its down to some other issue other than the rules themselves.

The processor utilization does'nt go above 60% when this is enabled and large mails are being processed, but memory effectively gets exhausted and the server basically hangs, the only way to resolve is to reboot the server. These servers have been running 100% up until last week. maybe a .net issue causing this now as that is the only thing that has been upgraded in the last week in these servers. Memory on these servers now stable at around 500Mb, spiking up to max 700Mb and then dropping again.
0
keith dovale Replied
Been running now all weekend without outgoing content filtering and NO further issues so far..
0
CLEBER SAAD Replied
I use a 15.3 latest version and the memory and processor become 100% top all time. With the 14.2 I don't have any problem.

The clamav it's disabled and there is no AV in this server. Same config of the latest 14.2 version.
0
Neal Culiner Replied
Is this custom content filtering per domain, per user, exactly where? Can you open a case with ST to get them to fix this?

Good work!
0
keith dovale Replied
This is based on custom rules in the main antispam section, we use content filtering on all our domains, but I have not seen any issues with that on a user or a domain. This only has been seen on the outbound filtering which we implemented about a month ago, as we have never seen a benefit to use it, however we enabled bayesian, custom rules, and remote spam assassin, and it is quite effective when forwarding mails from an inbound mail server to the user mailbox server, however the custom rules which catch a lot of unwanted spam causes this high spike of cpu it is not worth the backups it causes, we are using bayesian and remote spam assassin still, its about 50% effective than what it was, but the servers are running back to spiking 30% to 40% utilization which is fine with me.
0
Neal Culiner Replied
So in the Antispam Administration section just turn off the "enable for SMTP outgoing blocking"? Or is this also/or in the Advance settings area Abuse Detection? I had one item "message sniffer" enabled for outgoing blocking checked and then in SMTP blocking had Outgoing Weight Threshold checked. I'll try turning those off and cycle the service and see what happens.
0
Paul Blank Replied
Is it possible that the high RAM usage is a symptom, rather than a cause, of server hangs? Properly designed programs should swap to virtual memory when RAM usage is high, and at most slow down, as opposed to completely hanging.
 
Also, the server should report this in its event logs, at the least.
 
 
0
CLEBER SAAD Replied
Hello Paul,

In our case, the same server configuration from 14.2 and 15.3 (upgraded from 14.2). With the version 14.2 works very fine. With the version 15.3 the CPU and MEMORY growup very fast.

Exactaly same config.
0
Bruce Barnes Replied
Most current version is: 15.3.6109 . Try upgrading to that version, but be aware that doing even a minor version upgrade requires an uninstall, reboot and reinstall. Your data will be preserved, but make certain you enter any custom installation path (s). You must also disable the built-in SmarterMail web server, and run under IIS, preferably with TLS security properly configured. PM me if tou need assistance on setting up the encryption products for your server OS or SmarterMail ports.
Bruce Barnes
ChicagoNetTech Inc
brucecnt@comcast.net

Phonr: (773) 491-9019
Phone: (224) 444-0169

E-Mail and DNS Security Specialist
Network Security Specialist

Customer Service Portal: https://portal.chicagonettech.com
Website: https://www.ChicagoNetTech.com
Security Blog: http://networkbastion.blogspot.com/

Web and E-Mail Hosting, E-Mail Security and Consulting
0
keith dovale Replied
yup correct..
0
Merle Wait Replied
So I had same experience as keith dovale September 30 ... In that mem usuage was not so bad... but CPU usage suddenly would spike. I too traced it back to Spam Assassin being in conflict with SM (we are on 15.2).  Created a problem ticket with ST; they recommended going to 15.3)..  We did.. so far no more spiking... but it seemed to be very random.
 
We were able to capture the CPU spike and create a dump for ST. Not sure what, if anything, was found from that.
 
My only suggestion would be turn off (trial an error) a couple of other systems or functions... just to see if SM goes back down.   We are a small shop....  avg about 6K inbound daily.... mem=avg 600mb.   
0
Matthew Titley Replied
SM 15.2.6
Win2012 R2 (Hyper-V VM with dynamic memory, 4GB currently assigned, 4 vCPUs)
SM Service using 880MB
ClamAV using 450MB
Cyren using 40MB
Decludeproc using 16MB
About 500 users, 30K messages a day.
 

Reply to Thread