MailService.exe crashes so often

Problem reported by Stefano - 7/1/2019 at 6:49 AM

Resolved

Hi,

since one week, our server is so slow and in these last 3/4 days MailService.exe crashes more than 3 times a day.

Today it had just crashed for the third time, these are the latest two errors that we get.

Nome dell'applicazione che ha generato l'errore: MailService.exe, versione: 100.0.7117.30305, timestamp: 0x5d155639
Nome del modulo che ha generato l'errore: mscorlib.ni.dll, versione: 4.7.3416.0, timestamp: 0x5cabfaa3
Codice eccezione: 0xc00000fd
Offset errore 0x0000000000589bce
ID processo che ha generato l'errore: 0x80c
Ora di avvio dell'applicazione che ha generato l'errore: 0x01d52de2b9d51fd6
Percorso dell'applicazione che ha generato l'errore: C:\Program Files (x86)\SmarterTools\SmarterMail\Service\MailService.exe
Percorso del modulo che ha generato l'errore: C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\bef0a43af1bb9a52ee47a6f60bec2961\mscorlib.ni.dll
ID segnalazione: 6744e891-a423-4258-89d5-5b6d6da5d928
Nome completo pacchetto che ha generato l'errore: 
ID applicazione relativo al pacchetto che ha generato l'errore:

Nome dell'applicazione che ha generato l'errore: MailService.exe, versione: 100.0.7117.30305, timestamp: 0x5d155639
Nome del modulo che ha generato l'errore: clr.dll, versione: 4.7.3416.0, timestamp: 0x5cabfc63
Codice eccezione: 0xc00000fd
Offset errore 0x000000000006864e
ID processo che ha generato l'errore: 0x1c44
Ora di avvio dell'applicazione che ha generato l'errore: 0x01d52fd6fc9aed78
Percorso dell'applicazione che ha generato l'errore: C:\Program Files (x86)\SmarterTools\SmarterMail\Service\MailService.exe
Percorso del modulo che ha generato l'errore: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
ID segnalazione: 2c79b1f6-65e9-494a-acf2-1fbdf6067b6d
Nome completo pacchetto che ha generato l'errore: 
ID applicazione relativo al pacchetto che ha generato l'errore:

Someone has got the same problems?

We're working with SM17.7117.

34 Replies

Reply to Thread

Christian Schmit Replied

7/1/2019 at 8:03 AM

We are having one server where mailservice.exe is constantly crashing with build 7118. We had to go back to build 7082 which seems to be running stable again.

Stefano Replied

7/1/2019 at 8:22 AM

But older versions had a bug about the duplication of the appointments in the calendar.

I can't go back!

Kyle Kerst Replied

7/1/2019 at 9:49 AM

Employee Post

Hello and good morning. First, I'm sorry to hear you're experiencing server crashes and I would be happy to help get you back up and running. It is possible you are encountering an issue we identified in the previous release build, and we recommend you deploy the latest available minor update build 7118. Please perform a minor upgrade to this, and if you continue to see these issues post-upgrade; please submit a support ticket so that we can investigate further. Thanks in advance!

Kyle Kerst Acting IT Manager SmarterTools Inc. www.smartertools.com

Stefano Replied

7/2/2019 at 12:00 AM

Dear Kyle,

updated last night, this morning it had crashed again, ~~twice~~ four times.

Nome dell'applicazione che ha generato l'errore: MailService.exe, versione: 100.0.7118.38754, timestamp: 0x5d16e9bc
Nome del modulo che ha generato l'errore: clr.dll, versione: 4.7.3416.0, timestamp: 0x5cabfc63
Codice eccezione: 0xc00000fd
Offset errore 0x000000000006864e
ID processo che ha generato l'errore: 0x4d0
Ora di avvio dell'applicazione che ha generato l'errore: 0x01d530525e7f6c06
Percorso dell'applicazione che ha generato l'errore: C:\Program Files (x86)\SmarterTools\SmarterMail\Service\MailService.exe
Percorso del modulo che ha generato l'errore: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
ID segnalazione: f4e0cebd-7a80-4005-b2df-b79b98ee45d7
Nome completo pacchetto che ha generato l'errore: 
ID applicazione relativo al pacchetto che ha generato l'errore:

Nome dell'applicazione che ha generato l'errore: MailService.exe, versione: 100.0.7118.38754, timestamp: 0x5d16e9bc
Nome del modulo che ha generato l'errore: mscorlib.ni.dll, versione: 4.7.3416.0, timestamp: 0x5cabfaa3
Codice eccezione: 0xc00000fd
Offset errore 0x000000000057b6dd
ID processo che ha generato l'errore: 0x11d4
Ora di avvio dell'applicazione che ha generato l'errore: 0x01d530a2ff8f31a0
Percorso dell'applicazione che ha generato l'errore: C:\Program Files (x86)\SmarterTools\SmarterMail\Service\MailService.exe
Percorso del modulo che ha generato l'errore: C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\bef0a43af1bb9a52ee47a6f60bec2961\mscorlib.ni.dll
ID segnalazione: b52cee4a-2dfd-44ee-80bd-7aa7472f3a5f
Nome completo pacchetto che ha generato l'errore: 
ID applicazione relativo al pacchetto che ha generato l'errore:

We CAN'T GO ON like this, restarting the service three/four times a day!

I opened a ticket days ago, but we really need SmarterTools' help.

Kyle Kerst Replied

7/2/2019 at 3:39 PM

Employee Post

Stefano, are you seeing high CPU or memory utilization before the crashes occur? When the crash does occur; is the service still running or does it stop along with the crash? If you're seeing any repeatable pattern on these crashes - can we have you run dump on the service pre-crash and get this over to us so we can investigate further? Thanks!

Kyle Kerst Acting IT Manager SmarterTools Inc. www.smartertools.com

Kyle Kerst Replied

7/2/2019 at 4:21 PM

Employee Post

Quick update on the troubleshooting here! Support has been monitoring one of the servers involved in troubleshooting these issues for about 10 hours now without any further crashes. We do have ProcDump set up to trigger a crash dump on service failure, so we should be able to collect a good amount of diagnostic data should these issues occur again. Please keep us posted on any issues seen.

Kyle Kerst Acting IT Manager SmarterTools Inc. www.smartertools.com

Stefano Replied

7/2/2019 at 11:47 PM

Hi Kyle,

the CPU usage is quite always low, the memory utilization is medium/high but it's always like that.

We have also increased the RAM days ago, just to be sure about it.

Your colleague Rod had started a ProcDump on our server, but our server usually crashes during morning / afternoon here (UTC +2), so now it's still running but without any crash.

I will keep you updated about it.

Stefano Replied

7/3/2019 at 2:34 AM

Hi Kyle,

this morning at 10.36 italian time the server had crashed again.

The only thing I've changed is the settings of the service when it goes down, I've chosen to restart it after 0 minutes, so our mail server won't be down for a long time.

I think that the ProcDump didn't managed to do anything as you can see there

I've restarted the ProcDump, I will wait for your help, this error is giving us so many problems.

We can't go on like this!

Christian Schmit Replied

7/3/2019 at 3:07 AM

Today we had another server running build 7118 crashing.

We started procdump on this server and are waiting for the next crash to occur.

Stefano Replied

7/3/2019 at 6:49 AM

Few minutes ago the webmail wasn't working but the SmarterMail Service was up.

Stopped and restarted IIS, everything came back to work.

We need help and we need it fast.

EDIT: After few minutes, SmarterMail Service crashed again twice :(

Kyle Kerst Replied

7/3/2019 at 11:49 AM

Employee Post

Thank you Stefano. I've received word from Rod/Development that we have since implemented some additional debug logging that should capture more information if/when the crash occurs again. We'll keep you posted as we find out more.

Kyle Kerst Acting IT Manager SmarterTools Inc. www.smartertools.com

Stefano Replied

7/3/2019 at 12:16 PM

Hi Kyle, we hope to have an answer soon.

I know that tomorrow all the USA will be on holiday, so the time will not help us ;)

Employee Replied

7/3/2019 at 2:54 PM

Employee Post

Stefano, Christian,

We replied to your support tickets but wanted to post here for any other users that might see this. The SmarterMail Developers were able to determine what's causing these crashes. They've created a custom build that resolves the issue and allows for additional logging should any future mailbox issues occur.

Here is the custom build:

http://www.smartertools.com/downloads/SmarterMail/CustomBuilds/100.0.7123.24818/SmarterMail_7123.exe

To enable the custom mailbox logging after running this build, log in as the Administrator and go to Troubleshooting > Options. In the Debug Log IDs textbox, enter 111 and save.

Sébastien Riccio Replied

7/4/2019 at 6:37 AM

Hi Andrea, we also have the mailservice recently crashing multiplie times à day and we would like to try this custom build to see if it helps.

Is there a changelog of other changes/fixes from 7118 to this build available?

Sébastien Riccio System & Network Admin https://swisscenter.com

Employee Replied

7/4/2019 at 9:28 AM

Employee Post

Hi Sébastian,

I believe this is simply the fix to the mailservice.exe crashing. I'll double check.

Employee Replied

7/4/2019 at 10:13 AM

Employee Post

Here is the change log for the custom build.

FIXED: Note created date is not properly set nor synced from EWS clients.

FIXED: Scenario in which random messages may be deleted without a cause.

FIXED: Scenario where the default EndBy date goes a day ahead when creating an appointment.

FIXED: Primary system administrators are able remove certain permissions from themselves (all permissions should be enabled).

FIXED: Root webmail URL does not validate if a port number is included.

FIXED: MailService crashes often after updating to Build 7118.

Stefano Replied

7/5/2019 at 5:49 AM

Hi everybody,

after the update, MailService.exe has not crashed (until now)

But sometimes the webmail slows down, don't know why...

At least I don't have to restart the service many times ;)

Bruce Replied

7/5/2019 at 6:46 AM

Upgraded from 7090 to 7118 this morning and SmarterMail has started crashing.

Thanks for the custom build will give it a try and see if it solves the issue.

Employee Replied

7/5/2019 at 10:33 AM

Employee Post

Stefano,

We chatted about your webmail slowness in your ticket, but I wanted to post here as well. We found that one of your users had a mail folder with a trailing space at the end of its name. This caused a recurring serialization exception which, subsequently, caused some concern with your disk i/o. This could explain your report that webmail is running slowly. We've corrected the user and hope the slowness is mitigated. Please let us know.

Bruce,

We just released the custom build as a new release to the public, so you'll find it on the Downloads page. Please let us know if this build resolves the crashing issue. In addition to the upgrade, please log in as the Admin and go to Manage icon > Troubleshooting > Options tab. On the Log Files card, enter 111 in the Debug Log IDs (one per line) textbox. In a day or two, please download the 111 log for the time it's been enabled, and send it to us by starting a support ticket here: https://www.smartertools.com/account/#/support.

Nathan Replied

7/5/2019 at 1:41 PM

The crash sounds like an issue we logged a ticket for about a month ago. It turned out mailservice.exe crashed every time 1 specific ActiveSync user synced with their device.

We worked around the issue by deleting the device profile via webmail.

Are you running with ActiveSync enabled users?

Rod Strumbel Replied

7/8/2019 at 7:21 AM

If folders should not have trailing spaces... wouldn't it make sense to simply TRIM the folder names before creating them (assuming they don't duplicate something else?) Trying to get users to pay attention to such necessary rules is fruitless.

Sébastien Riccio Replied

7/8/2019 at 1:45 PM

It makes sense to trim white spaces left and right n folder names. Mail clients should apply this too.

Sébastien Riccio System & Network Admin https://swisscenter.com

Employee Replied

7/9/2019 at 2:16 PM

Employee Post

In recent versions of SmarterMail we do trim the folder names before they are created. We suspect that this was a lingering scenario from before that change.

Sébastien Riccio Replied

7/11/2019 at 11:25 PM

Sorry to raise this topic again, but following the multiple crashes we had, some of our customers reported their account wasn't accessible anymore. After digging into the user folder on the server I noticed a pattern that all these unavaible accounts were missing settings.json file but instead there was a settings.json.tmp in the folder !

Renaming the file to settings.json recovered the account.

It's the first time we see this happenning, is this related to the crashy SM build ?

At least the settings.json.tmp file had a filedate when lot's of crashes occured.

Has anyone else found settings.json.tmp files in their user dir (and missing settings.json).

Anyway, whatever is the source of the problem, SmarterMail should check at least at startup for settings.json.tmp files and rename them... Or we need to script ourselves a find/rename script we have to run after each crash so we avoid user complaints ?

Thanks

EDIT: I did a wider search for *.tmp, and found out there was also gal.json.tmp files (and missing original gal.json) files in some of our users... With same date as when the crash occured... This is really worrying!

Sébastien Riccio System & Network Admin https://swisscenter.com

Stefano Replied

7/12/2019 at 12:02 AM

Hi Sébastien, we have had the same problem.

I think that something about this crashes doesn't write these files in the correct way.

With the latest version of SmarterMail, we didn't had crashes anymore and everything seems fine ;)

Sébastien Riccio Replied

7/12/2019 at 1:23 AM

Hello Stefano,

Thank you for the feedback.

Yes since latest SmarterMail versions we have a lot less crashes. It was more than 10 times a day and now it's about 1 time per day.

Also the tmp files we found are all dated from the constant crash period. But still it would be a good idea that SmarterMail can automatically recover from this, at the next startup.. :)

Sébastien Riccio System & Network Admin https://swisscenter.com

Stefano Replied

7/12/2019 at 5:08 AM

Sébastien, which version have you installed on your server?

Sébastien Riccio Replied

7/12/2019 at 6:59 AM

Hi Stefano, 7123, the custom build of this thread. Not yet updated to the latest public as there are no other important changes listed. Or is there ?

We try to avoid updating too often and interrupt service for our customers.

Sébastien Riccio System & Network Admin https://swisscenter.com

Stefano Replied

7/12/2019 at 7:01 AM

So strange :/

I think you should open a ticket, that's the only way to get rid of this annoying problem.

Sébastien Riccio Replied

7/12/2019 at 8:23 AM

Yes Stefano, ticket is open :)

Kind regards

Sébastien Riccio System & Network Admin https://swisscenter.com

echoDreamz Replied

7/12/2019 at 11:55 AM

I assume when SM crashed, it was writing the new settings.json file as the .tmp file, the crash of course halted the action leaving you with the .tmp. SM should check for these on startup and apply the correct action.

Sébastien Riccio Replied

7/12/2019 at 2:26 PM

Yes echoDreamz, I just wonder what the original settings.json file is removed. Why not "simply" create a .tmp and then rename/overwrite the original file with the changes. At least the original file would still be here in case of faillure.

I mean on linux i would echo new settings to bla.json.tmp

then "mv bla.json.tmp bla.json"

No risk then to have a missing json file...

Anyway let's hope the case will be handled gracefully in the future :)

Sébastien Riccio System & Network Admin https://swisscenter.com