Back to Community Threads
23
Allow reporting of Spam via IMAP
Idea shared by Colin M - 9/10/2014 at 11:10 AM
Declined
Currently SmarterMail will create a .eml file for any emails that a user manually marks or unmarks as spam via the web interface for the purpose of training the bayesian filter. These are stored in Service/Spam|Ham/Type[123]. The details of this are explained here. This is wonderful except for the fact that a *lot* of people never use the webmail! (about 1% of my users use webmail)
 
I see no reason that the same couldn't be done for email users via IMAP. That is, when a MOVE command is issued for a message into the "Junk E-Mail" mailbox consider this the same as a user clicking Junk from the web interface. Similarly, when a message is moved out of Junk-Email into any other folder besides trash (or even just the Inbox) it is considered Ham.
 
POP users are SOL..

32 Replies

Reply to Thread
1
This is a good idea and I think I have read people asking for it before.
 
But I question the effectiveness of the bayesian filter anyhow... Like I give mine a score of 8, which alone will not actually do much.
0
I don't use the bayesian filter at all, but having these user-marked spam in a folder that is easily accessible would make it easier for building blacklists for MTAs, relays, URLs, etc.. Also would make it easier for me to see what spam is getting through for other users as currently I only look at my own spam when tuning rules.
0
And if not for the entire domain, at least for the specific user who is training their own bayes - it can improve their spam filtering.
0
I second this. It really is a needed feature. Nobody uses webmail here, we all use IMAP. Bayesian filter should be able to learn from what we moved to "Junk E-Mail" folder throught IMAP. It'd be great if we could mark which folder SM will look for learning what's spam (for us it's the "Junk E-Mail" but others may not use the same, this would best be configurable).
0
While this is a great idea, will this cause an issue with the IMAP protocol, IE:
 
If SmarterMail implements this capability within the SmarterMail program software's IMAP protocol, will devices and clients which are connected via IMAP be able to take advantage of the new code, and;
 
If SmarterMail implements this capability will it cause other issues with the overall IMAP protocol, potentially causing other issues?
Bruce Barnes ChicagoNetTech Inc brucecnt@comcast.net Phonr: (773) 491-9019 Phone: (224) 444-0169 E-Mail and DNS Security Specialist Network Security Specialist Customer Service Portal: https://portal.chicagonettech.com Website: https://www.ChicagoNetTech.com Security Blog: http://networkbastion.blogspot.com/ Web and E-Mail Hosting, E-Mail Security and Consulting
0
I don't see why it would. This is not a change in IMAP protocol. It's just what SM could do once it detects that an item was moved from folder Inbox to folder Junk. Has nothihng to do with IMAP per se, but the fact that many people use IMAP to access their mailbox limits them in their ability to "mark spam", because the only way to mark spam in SM is through the webmail interface. We want SM's bayesian filter to be able to actually learn from the items we moved to junk folder, and not learn only via the "mark as spam" action in webmail.
0
Actions are there (to manually mark a certain message as spam), the only addition is what triggers the action. The existing trigger is the "mark as spam" action in webmail. We just want to add another trigger. The IMAP MOVE command (when something is moved say from any folder to a designated SPAN-learning folder, that folder should be configurable) - should be interpreted the same way as "mark as spam" action in webmail interface. That's it. It's actually a (small) feature request. Shouldn't be hard to add and would save us a lot of time.
0
The question is not whether it can be done, but whether clients and devices will actually transmit the command back to SmarterMail so it is properly executed.
Bruce Barnes ChicagoNetTech Inc brucecnt@comcast.net Phonr: (773) 491-9019 Phone: (224) 444-0169 E-Mail and DNS Security Specialist Network Security Specialist Customer Service Portal: https://portal.chicagonettech.com Website: https://www.ChicagoNetTech.com Security Blog: http://networkbastion.blogspot.com/ Web and E-Mail Hosting, E-Mail Security and Consulting
0
I'm talking about what can trigger an action. In this case, it has nothing to do with clients and devices. IMAP is a standard interface, on the "outer part", the protocol side that is visible to the world - that interface should look the same. The change is on the "inner part", so IMAP (or any other protocol) shouldn't actually change. If we were to look at network packets from and to clients there should be no change at all on what goes over the wire. I am talking about what triggers the action to mark something as "this is actually spam, please pass it to bayesian so it can learn from it". A trigger could be an IMAP MOVE command - that's an actual command, that already exists, and should not change on the protocol side that speaks with clients. I'm not sure how to explain this any better...
0
Bruce, looks like I was a bit quick to comment. I don't actually know if IMAP implementation in SM supports the MOVE command because I just assumed from the original post. Just checked and found out that it's an extension, more info here: http://tools.ietf.org/html/rfc6851 I also don't know how many IMAP clients support that MOVE command. Maybe it's common and maybe not. So you may be right when you say that it depends client app. Maybe someone knows better if this extension is regular in IMAP implementations or is something generally not present, before jumping to conclusions. Maybe you know the answer to this. In any case, there are several options as I see it: a) you could add the feature through the MOVE command (so IMAP clients that support the MOVE command could benefit from it); b) you could just add a different "trigger" (say it could react in response to a COPY command - whatever gets copied to the designated Junk folder can trigger the action, that should work in all IMAP clients hopefully); or c) implement both and make it configurable, so we can choose what method(s) trigger the action
0
Without firing up a telnet session (actually openssl s_client - security!!) I can't say for 100% certainty, but I am otherwise 99% certain that the MOVE is supported in every IMAP client I've ever used and by every IMAP server I've ever used including SmarterMail.
0
Sound good enough to me :) So, Colin, now that there's more material, how do we make a feature request ? Besides chatting here on community forum.
0
When you find that out, David, please let me know. :)
1
I know this thread is a few months old, but I have a suggestion that might help everyone out, POP, IMAP and WebMail users. Why not have SM have the capability for end users submit spam email to a specially setup email address for the spam filter listen in on and learn from?
 
For example, spam@local.com or whatever mail address you want to setup for the spam filter to listen for. Then the user forwards the spam to that address. Any submissions to this address from local users would be recognized as a spam report and train the Bayesian filter accordingly. Allow this to be a global setting for all domains and only allow local authenticated users to use it. It's a win/win for everyone. Not only POP, IMAP but also Webmail users could take advantage of it. That way it's universal for everyone.
0
Unfortunately that won't work because when the email is forwarded by default most email clients strip all of the important headers that show where the original came from. You can try to instruct users to "forward as attachment" but in my experience I have to remind most users of this every time. Even then I'm not sure if all of the headers are intact...
0
The previous product we used worked that way before we handed spam filtering over to SM. A user would need to forward a message as an attachment and the filter would start to trap those messages. So it all comes down to training your users how to submit spam and yes the headers are all intact on a forward as attachment option. So I really think this is something SM development should think about implementing. It really would benefit everyone and make spam filtering that much better.
3
A few months old?!?!  This was suggested way back in 2014. :-)
 
It's also on the top 10 list of most-requested enhancements.
https://portal.smartertools.com/community/search.aspx?sortBy=1&tagID=3
 
Let's hope ST understands what their customers want and switch this Idea to Planned or In Progress.
0
I realize maybe I should have specified, the last time someone posted on this was a few months ago. But yes ST really needs to address this. My guess is that they just assumed all users were webmail and no one uses email clients. That's not the case, in fact it's the other way around. People are used to using a mail client like Outlook, etc.
0
Employee Replied
10/4/2016 at 10:46 AM
Employee Post
Hi,
 
For the SmarterMail 16 release we are just about exclusively focused on the new interface.  Re-writing that from scratch is a very large task and taking up most of our development time.
 
The release after that should be a large feature release where we can do many requests like this one.  Since we haven't planned out SmarterMail 17 yet I can't say for certain that this will be in it, but it likely will due to the number of requests it had received.
0
Awesome! Thanks, Bryon.
0
Thank you for the acknowledgement Bryon. We appreciate it.
0
Thanks for the update, Bryon. I'm curious though how it was determined that a web interface rewrite was such a high priority over what sounds like just about everything else? Do people complain much about it? Just a data point: I'd say about 95% or more of our users use IMAP and maybe 1% use EAS (we only added recently) leaving only a couple percent for Webmail and of those I never hear any complaints except from my wife if she loses a composed email due to a session timeout, lol.
1
I asked the same question back in February...
https://portal.smartertools.com/community/a87523/why-new-ui-in-smartermail-16.aspx
0
Employee Replied
1/25/2017 at 4:10 PM
Employee Post
Hello everyone,
 
After much consideration, we have determined this functionality is not on the roadmap for future versions of SmarterMail. We do appreciate your feedback, and thank you for your participation on this thread.
0
Employee Replied
5/9/2018 at 5:18 PM
Employee Post
UPDATE: After the removal of Bayesian filtering, this feature has been brought up for discussion once again. After much back and forth, it's been decided that this functionality will not be added into SmarterMail. Please understand that with the removal of Bayesian filtering, SmarterMail will no longer contain traditional mark as spam functionality. Instead, users will block unwanted senders using new spam filtering options or use content filtering to manage unwanted emails. We are discussing the changes to current functionality for managing unwanted email on the thread linked below.

In short, every email client treats spam differently, and there are many pieces within SmarterMail to consider. We unfortunately can't make the assumption that a user who moves a message into their client's junk/spam folder knows that all future messages from that user will be blocked. The features available in a user's preferred email client for managing unwanted emails (whether that's block sender or mark as spam) will remain in that client alone.

DISCUSSION: Block Sender / Mark Spam Functionality: https://portal.smartertools.com/Main/frmThread.aspx?threadid=90628
0
One last follow up to this; since bayesian has been removed, why is the Ham\Type 3 folder still filling up with messages? Just curious. If bayesian is no longer implemented I don't see the need to waste CPU on doing that particular task. :)
0
Employee Replied
7/3/2018 at 10:38 AM
Employee Post
Hi Andrew. We have no code in place anymore that directly does anything with those particular spam/ham folders. I believe MessageSniffer modifies the contents of those folders. ClamD and Cyren would not.

I noticed you're not utilizing Message Sniffer, so that folder should not be filling up with messages. If you would like, I can help you get in touch with the Support Department for their review. You'd need to submit an Email or Phone Ticket so that the agent can get access to your server. (Just keep in mind that the ticket will likely be refunded back to your account since it would likely be a bug that's causing this.) Let me know if you'd like me to submit that ticket on your behalf.
0
I know this is an old thread, but being able to easily flag something as spam from an IMAP client really is a valid idea. As with others who have posted, IMAP is my primary use case. I only touch the web interface for administration and occasionally to mark things as spam. 

I don't understand why there has to be any client side support for this either. Simply create a 'system' folder within SmarterMail, that shows up in any and every client, and for every user, as a standard IMAP folder. Name it 'SPAM Learning' or something. Now a user needs only to drag an email into the folder. Any email that lands in the folder gets immediately moved to a processing folder (so that it disappears from the users mailbox and client) then gets marked as spam, blocked, etc.

This was a feature in a really old mail server I used around 15 years ago, I think i was MDaemon or maybe True North Internet Anywhere mail server.

It seems to me that spam prevention is really a key feature in a mail server, and making it as easy as possible for non-technical end users is just common sense.I can't really wrap my head around why there is any resistance to what is clearly a needed feature.

Scott
1
Here's what I have done to work around the issue at hand with newly reported spam by our users, I created an account on our domain called spam@whateverdomainyouareusing.com and instruct our users to forward the message as an attachment to that account. Then once they do that, I have all the original headers intact from the original mail. I log into the SM web interface in that account and download the reported spam message attachment. Then I transport the email to our server at the console and then train our external spamassassin installation manually using sa-learn to learn that message as spam.

I realize this is a long process, but it's the only way I can make sure we are getting the latest spam emails trained into our external SA installation since the removal of the Bayesian portion in SM.
1
...users to forward the message as an attachment to that account...

This is the part that never happens. Users are too lazy if there's more than half a click involved.

Drag to a folder - simple

remember to forward as attachment?
remember where to find that damn button because I don't use it any other time?
remember some email address to send it to?
eh, too much work, why bother? the IT guy will figure it out by magic without my help. 

at least that's the mentality of the folks I work with.
1
Yep, we tried the spam@ thing, users dont care, too much work like you said Scott. Move an email to a folder, drag, drop done.
0
Our users are half and half. Some of them listen and some of them don't bother. It's always that way.
Back to Community Threads

Reply to Thread

Related Knowledge Base Articles

Recommended SPAM SettingsSmarterMail > Antispam and Antivirus
Cannot Send Email MessagesSmarterMail > Troubleshooting
Configure Perfmon to capture Disk IO activity and potential Disk Issues.SmarterMail > Troubleshooting
Find a Compromised AccountSmarterMail > Troubleshooting
Messages to Yahoo! Emails are Temporarily DeferredSmarterMail > Troubleshooting