Hi Joshua,
When testing in version 13.0, I see that the Password Policy Compliance list is automatically and immediately updated when I change the password requirements. I believe this should function the same way in older versions as well. Which version are you currently running?
One thing you might do to test out whether this is working correctly in your version is change your Minimum Password Length to something very large, such as 25. Click Save, go back to the Manage section and make sure you click the Password Policy Compliance button in the navigation pane again (this will refresh the list). You could also go a step further and create a test user with a password that does not meet the requirements (you'll be able to do this if you're logged in as the System Admin, as password requirements do not apply to Sys Admins). Then check back on the compliancy list to see if that user is listed.
I would suspect that increasing the password length to that high of a value would show a large jump in non-compliant users. It did for me when testing it out. Please let me know what you find. If there is no change in the number of non-compliant users I would recommend opening a support ticket so we can look into this more for you as this is not expected behavior.
As a side note, in the latest major release, SmarterMail 13.0, we added new password policy features that you may be interested in. Version 13.x now includes the ability to set a password expiration, notify users when their password needs to be updated, and disable outgoing SMTP for users in violation.