Trying to make sense of this problem and its connection to OAUTH2. The big surprise is that Outlook tries to log into Active Directory before it tries to login into the mail server. But the desire to support OAUTH2 seems to explain why.
My layman’s understanding of OATH2:
- Trusted device is authenticated to the server
- A trusted application is used on the trusted device
- If the device or trusted application trusts the user, then the server chooses to trust the user also.
Applying this scenario to Outlook:
- The trusted device is the Windows PC and the trusting server is Active Directory.
- Outlook can only become a trusted application if it authenticates to the same server context as the PC, so it must also authenticate to Active Directory.
- A design assumption is that Outlook has been configured with an application password that uniquely identifies the application (probably based on user-agent string) and a specific user account.
- Outlook tries to determine the Active Directory account that will work with the configured email address and application password. (Outlook apparently ignores the email address field in Active Directory.) Outlook first tries for an exact match on the full email address. If that does not exist, it settles for an exact match on the username portion of the email address in any domain. If the second guess is attempted to a server where the same username exists in multiple domains, results are likely to be unpredictable and undesirable.
- If no Active Directory account is found, Outlook gives up on OAUTH2 and uses NTLM.
- If an Active Directory account is found, Outlook attempts multiple logins, triggering the login failures. After enough failures, it finally gives up on preparing OAUTH2 authentication and uses NTLM.
I assume that Office365 provides a way to configure an application-specific password for Outlook into the user’s Active Directory account. I don’t use Office365 and I do not know how to do that in a fully on-prem environment.
Therefore, the workarounds can be:
- Find a way to disable Outlook’s OAUTH2 behavior completely.
- Find a way to implement an application-specific password in your Outlook/ActiveDirectory environment.
- Use any user interface other than OAUTH2-enabled Outlook.
- Ensure that Outlook finds an Active Directory account where its store password will work. This will occur if:
- the email address is resolved to the correct Active Directory account and the mail system authenticates against that same Active Directory, or
- the email system is configured to use the same password as the Active Directory account that Outlook finds.
- Ensure that Outlook finds an exact-match account which is already is disabled, so that it eventually gives up trying to enable OAUTH2. This is the strategy that Nick described.
Implications for SmarterMail
If a SmarterMail account is configured for Active Directory authentication, then I expect SmarterMail to be configured with the same Active Directory account as the one that Outlook finds. SmarterMail already allows the Active Directory domain and user to be different than the email address.
If a SmarterMail account is configured to use SmarterMail authentication, or any security environment other than the PC’s Active Directory environment, then the problem can occur, and one of the workarounds will be necessary.