I have gotten very opinionated on the topic of what is needed in a spam filter. It is much more complex than a single feature request.
It was not clear if you have a single-box solution. I strongly recommend configuring an incoming gateway to filter your inbound mail. Having local and remote mail arrive at the same place will make spam filtering and log analysis much harder.
Spam filters can be configured in series. I use three different servers in front of my SmarterMail server. I think it is comment to asssume that all spam filtering must be done in one device.
Overall, SmarterMail's strength is email, not spam filtering. I don't think anyone can wait while they build expertise and create code to do what others have been attempting for years. Nothing wrong with what they offer, it is simply not enough.
Fortunately, SmarterMail offers a free incoming gateway option and that configuration is extensible. I opted to use SmarterMail with Declude as my initial extension. Then I used custom filtering within Declude to enhance that product further. Open-source Python code gave me an improved SPF and added DKIM signature verification. Additional effort moved some of the filtering into a SQL backend. After Declude filters out garbage based on source reputation, my commercial spam filters do the one thing they do well - complex content filtering.
Architecture - what you need
Email source analysis should involve all of these attributes:
- Source server, identified by Source IP, Helo name, and Reverse DNS name. The DNS names can be considered verified if they forward-confirm to the Source IP.
- Source mail system, identified by the SMTP Address and potentially verified by SPF linking the address to the Source IP
- The From header address, verified by alignment with an SPF-verified SMTP address or alignment with a verified DKIM signature.
(I have looked at a bunch of products that do not support all of these filtering criteria.) Operationally, some policies can be applied to a single attribute, but many policies need to be applied to sets of attributes. Examples are IP-SMTP, SMTP-From, DNSdoman-SenderDomain.
For mass mailers, the SMTP address will be the mailer and the From address will be the client. One of the biggest mass mailing organizations is serving both legitimate senders and criminals. So I filter on the SMTP-From domain combination. Some pairs are whitelisted, some are blacklisted, and anything uncategorized gets quarantined. I wonder how many ransomware attacks have succeeded because of spam filters that do not provide this configuration option.
In rare cases, two mass mailers are involved. One is indicated by the SMTP domain. The second one is indicated by the From domain, and the actual client is indicated by the username portion of the From address.
For spam sources, blocking on DNS host name is pretty powerful. Blocking simultaneously on host names, IP address, and domain names is even better. Sometimes the HELO name is determinative, sometimes the Reverse DNS name is determinative, so I always check both against my DNS domain reputation lists.
Quarantining messages with UTF-8 encoded subjects will block a lot of spam and a lot of advertising, much of it unwanted. (Personal messages in English will rarely have any encoding.) UTF-8 filtering will also generate a lot of false positives, so if you use this idea you need to be willing to work your quarantine. You also need a filtering engine that can express the exceptions that are needed.
Unfortunately, all of the above is only examining the last-hop server. This approach is only effective if the last hop server is an agent of the message source. Once a message is forwarded, the reputation of both good and bad message sources will get hidden behind the reputation of the forwarding system. So forwarded mail is really scary. Fortunately, I don't have much forwarded mail. The solution is to evaluate the entire header structure to identify the true sender of a forwarded message. Doing so is (a) complex, (b) uncertain, and (c) resource intensive. Solving that part of the problem an ongoing research interest.