We upgraded to 16.3.6885 from 16.3.6841 (we've gone back to a Monthly scheduled maintenance window to try and bring our Mail Server's annual uptime ratio back up to snuff so we were a couple minor versions behind until yesterday).
It would appear that 6-8 hours after the last server reboot everything stabilized and is working normally again. We have had no new reports of webmail issues since early this morning, and are no longer able to duplicate the issue from a dozen different devices on three distinct external networks. Chrome and Firefox Consoles (both with Caching on and Caching off) are not logging any significant errors other than numerous repeats of the following minor errors and warnings:
Failed to load resource: the server responded with a status of 400 () /api/v1/mail/folders:1
Failed to load resource: the server responded with a status of 400 ()
/api/v1/settings/shared-resources:1
[Violation] Added non-passive event listener to a scroll-blocking 'touchmove' event. Consider marking event handler as 'passive' to make the page more responsive. See https://www.chromestatus.com/feature/5745543795965952 angular-v-16.3.6841.24538.8d6225dcb713200.js:3
[Violation] 'setTimeout' handler took 55ms angular-v-16.3.6841.24538.8d6225dcb713200.js:55
[Violation] Forced reflow while executing JavaScript took 35ms angular-v-16.3.6841.24538.8d6225dcb713200.js:55
[Violation] 'load' handler took 180ms angular-v-16.3.6841.24538.8d6225dcb713200.js:111
[Violation] 'requestAnimationFrame' handler took 62ms vendor-v-16.3.6841.24538.8d6225dcb713200.js:101
(The first two of those Errors appears normal for users logged on as a Server Administrator.)
Early this morning, before things stabilized, we were logging the following Console errors at least every 14 minutes (sometimes every minute):
WebSocket connection to 'wss://smartermail.scarabmedia.com/signalr/reconnect?transport=webSockets&groupsToken=...&clientProtocol=1.5&connectionToken=...&connectionData=%5B%7B%22name%22%3A%22dashboardhub%22%7D%2C%7B%22name%22%3A%22mailhub%22%7D%5D&tid=1' failed: Error in connection establishment: net::ERR_NETWORK_CHANGED
start @ vendor-v-16.3.6841.24538.8d6225dcb713200.js:418
WebSocket connection to 'wss://smartermail.scarabmedia.com/signalr/reconnect?transport=webSockets&groupsToken=...&messageId=...&clientProtocol=1.5&connectionToken=...&connectionData=%5B%7B%22name%22%3A%22dashboardhub%22%7D%2C%7B%22name%22%3A%22mailhub%22%7D%5D&tid=6' failed: Error in connection establishment: net::ERR_SPDY_PROTOCOL_ERROR
vendor-v-16.3.6841.24538.8d6225dcb713200.js:418
WebSocket connection to 'wss://smartermail.scarabmedia.com/signalr/reconnect?transport=webSockets&groupsToken=...&messageId=...&clientProtocol=1.5&connectionToken=...&connectionData=%5B%7B%22name%22%3A%22dashboardhub%22%7D%2C%7B%22name%22%3A%22mailhub%22%7D%5D&tid=4' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
start @ vendor-v-16.3.6841.24538.8d6225dcb713200.js:418
WebSocket connection to 'wss://smartermail.scarabmedia.com/signalr/reconnect?transport=webSockets&groupsToken=...messageId=...&clientProtocol=1.5&connectionToken=...&connectionData=%5B%7B%22name%22%3A%22dashboardhub%22%7D%2C%7B%22name%22%3A%22mailhub%22%7D%5D&tid=10' failed: WebSocket is closed before the connection is established.
stop @ vendor-v-16.3.6841.24538.8d6225dcb713200.js:418
/signalr/negotiate?clientProtocol=1.5&connectionToken=...&connectionData=%5B%7B%22name%22%3A%22dashboardhub%22%7D%2C%7B%22name%22%3A%22mailhub%22%7D%5D&_=1541720342025:1
Failed to load resource: net::ERR_CONNECTION_REFUSED /signalr/negotiate?clientProtocol=1.5&connectionToken=...&connectionData=%5B%7B%22name%22%3A%22dashboardhub%22%7D%2C%7B%22name%22%3A%22mailhub%22%7D%5D&_=1541720342025:1 vendor-v-16.3.6841.24538.8d6225dcb713200.js:418
So, not only is Chrome and Firefox doing fine, but now we are also unable to duplicate the issue our users have been reporting with Safari 12.0.1 (14606.2.104.1.1) on Mac OSX Mojave v10.14.1 with it logging them out of SM immediately upon any browser activity whereas yesterday and early this morning we were able to duplicate it.
*SHRUG*
I do seem to recall another SM minor version upgrade issue like this from earlier this year where everything fell apart initially for a few hours after uninstall old SM/reboot/install new SM/reboot but returned to normal after an 8 hour delay.
So all's well that ends well? Nothing more to see here? Guess I'll just move along...and next month I'll be sure not to panic and always remember my towel.