2
MAPI stopped after reboot
Problem reported by Josip Meglaj - 2/1/2022 at 9:47 AM
Resolved
Hello,

I've upgraded the operating system from Server 2012R2 to 2019, and have now IIS 10.0 instead of 7.5 (its about time).
Migration was flawless as it should be, server were offline under 30 mins.

C: is OS and Smartermail Program Installation (NVMe)
D: is SmarterMail data Installation and spool files (NVMe)
E: is for customer / domain 1 (SSD)
F: is for customer / domain 2 and up (SSD)

Storage System of the domains "idles" at around 1-8MBps reading when syncing via MAPI.
Active Time around 1-2.5% (SSD). the NVMe idles at under 0.5% time.

Yesterday Monday, everything worked fine as it should until around 16:00, then MAPI stopped synchronizing on domain 1.
Today Tuesday, MAPI stopped synchronizing on domain 2 on around 10:15, after a reboot of the server.

This means, I have to resynchronize Domain 2 too from scratch.
Tried "Resync User Protocols", but no new E-Mails came into Outlook 2019/365, just the old ones.
Deleted Outlook OST File, E-Mail synchronization begins from scratch.

For me, these are annoying issues:
- why stopped MAPI on domain 1 - nothing into the logs
- why stopped MAPI on domain 2 after a simple reboot (no Windows Update, just a reboot)
- why cannot the server use more CPU cores (runs on 50-60% CPU) when under heavy load

Furthermore, how much RAM is recommended with MAPI, because I see now huge RAM usage levels during the synchronization of 88-92% which is unusally high (7.3GB of 8.0GB, 4.8GB is SmarterMail Service alone).

I have plenty of ressources, but now I'm afraid to shut down the Server and allocate more RAM due to the incident this morning.

Unfortuanlly, the MAPI mailboxes are huge: 60 / 57 / 13 / 11 / 10 / 9  / 6 / 5 etc. GB which makes the download even more annoying.

Does just I have this problems or do my customer use too big Mailbox sizes for Outlook or SmarterMail?

Webmail runs flawless, just MAPI doesn't run smooth as it should.

Thank anyone for additional information for optimizing my installation.

Regards,

Josip.

3 Replies

Reply to Thread
0
Josip Meglaj Replied
OK, hear me out, I wouldn't believe this myself If I didn't had the evidence:

Somehow Windows Server, specifically svchost.exe decided to change the System Time 745 hours in the future, and I believe (not sure) that this was the root cause for my first MAPI Problems yesterday (Message in German):

*******************************************
Die Systemzeit wurde von ‎2022‎-‎01‎-‎31T14:31:24.191880300Z in ‎2022‎-‎03‎-‎03T15:31:23.661228700Z geändert.
Änderungsgrund: An application or system component changed the time.
Prozess: '\Device\HarddiskVolume3\Windows\System32\svchost.exe' (PID 3152).
*******************************************

and \Device\HarddiskVolume3 is my System Drive:
DevicePath               DriveLetter
----------               -----------
\Device\HarddiskVolume3  C:        
\Device\HarddiskVolume6  D:        
\Device\HarddiskVolume8  E:        
\Device\HarddiskVolume10 F:        

In conclusion, the 745 hours time offset, caused from svchost.exe seems to be the root cause of this huge MAPI trouble.
Would be nice if someone from SmarterTools could confirm this.

To be honest, I have no clue how and why Windows Server had to change the time 745 hours into the future.

I've now set different time servers in Windows, because I'm not sure if it was the default "time.windows.com" which caused this:

w32tm /config /update /manualpeerlist:"pool.ntp.org time.windows.com time.apple.com time.cloudflare.com"
w32tm /resync
w32tm /query /peers

Regards,

Josip.
0
Kyle Kerst Replied
Employee Post
Thanks for this breakdown Josip, the time change issues would definitely cause your sync to get out of sorts, and likely explains the problems you were having there. Once the time is corrected please give the resync user protocols option once more and let us know if you continue to have sync trouble with this client. As to the root cause, this time change behavior is something I've seen in one other environment where the time was changed erroneously via NTP updates, and so I've typically recommended to my users that they leverage manual time settings since time changes can be extremely detrimental to sync and mail servers in general. 

On the memory usage, please note that MAPI is a heavy resource user, and is designed to use the memory available to us to provide the best and fastest sync and overall experience for those users. With that being said, we would be happy to take a look for you and help confirm whether the resource usage you're seeing is reasonable or not. Please feel free to submit a review ticket with us and we'll help get to the bottom of it for you. 
Kyle Kerst System/Network Administrator SmarterTools Inc. (877) 357-6278 www.smartertools.com
1
Josip Meglaj Replied
Marked As Resolution
Hello Kyle,

Thank you for your information, I've created a "normal" ticket 35E-298C10D6-0B2B and this can be closed..
The conclusion is the same: time change of 2682000 seconds is exactly one full month plus one hour (31d+1h), so I think that it was either a bug in Windows Server or the default NTP Server time.windows.com had an error.
My resolution to check several NTPs should be enough, or as you mentionned change to manual/no update.

The "resync user protocols" didn't work out, I waited a couple of minutes on one mailbox, had 40mbps traffic, but new E-Mails didn't show up. After 10 Minutes or so, the user was impatient and I performed the quick-and-dirty solution: delete the .OST file, located in %LOCALAPPDATA%\Microsoft\Outlook and the download began from scratch.

At the moment, I will not interrupt the synchronisation with the remaining users.
But on weekend, I will definitively upgrade memory from 8 to 12 or 16GB.

Regards,

Josip.

Reply to Thread