Amend Greylisting implementation to handle SMTP 'clusters'
Idea shared by Antony - October 16, 2015 at 2:17 PM
Under Consideration
Issue: Email providers the like of Office365/Outlook.com, SendGrid and Google are using different SMTP servers each time they attempt to send a message.  The result is that the email is delayed for longer than originally set in the SM greylisting settings.  We have tracked examples with the above providers where a message has been delayed up to 24 hours as a result of this.
 
Proposed solution: On the attempt after the expiration of the delay. If the IP/Sender combination isn't the same as the initial one, then check SPF/Sender combination and if SPF passes allow the email.  If no SPF then stick to the IP/Sender combination.
 
MDaemon can do this and we have seen the speed of delivery increase from providers that use different SMTP servers to send without a change to the amount of spam getting through.

30 Replies

Reply to Thread
0
I add the IP ranges of google and hotmail as safe (white list)
0
Yes, I also vote on this one.

Google now with the IPv6, add's outbound servers every minute, this are the last week's one's:

2001:4860:4000::/36
2404:6800:4000::/36
2607:f8b0:4000::/36
2800:3f0:4000::/36

And today already added this to the white list:
2a00:1450:4000::/36
2c0f:fb50:4000::/36
2607:f8b0:4002:c07::/36
2607:f8b0:4003::/36
6
This is now a major problem for us as well. Anyone have a solution to keep Greylisting but deal with email providers such as this?
0
We're having similar issues. What's the best answer?
8
I'm going to bump this.  We just had this issue with a client.  They had someone send email from outlook.com and it bounced around for over 1 1/2 hours.  Is there a way to maybe instead of using IP's to use the reverse DNS (confirmed with forward DNS) and failback to IP? and consider CIDR grouping instead of single IP's..  (Allow us to set minimum CIDR span a IP must reside in..)
 
Example.. 
mail-bl2nam02on0056.outbound.protection.outlook.com hits the server.. then...
mail-co1nam03on0058.outbound.protection.outlook.com hits the server since outlook.com is already in the accepted list.. then it's good to go.  
 
This is becoming a large issue.
7
As an (unfortunately manual) workaround - here is a list of MS provided SMTP server IPs for O365 which can be used to populate your whitelists.
 
 
<rant>
We have had to manually maintain whitelists for outlook.com to get around this greylisting issue  as well as a seperate whitelist to handle SMTP relay issues as described in this thread
 
 
Whitelists are not simply maintainable in this day and age, let alone having to maintain two different whitelists to address two different (yet similar) problems.
 
One would think that this type of list could be programmatically determined by looking up SPF records for a few key hosts (Outlook.com, Gmail.com, InsertYourFavoriteHostHere.com) and automatically whitelisting those IPs if a setting is enabled to do so.
 
If SmarterMail is not in a position to dedicate development bandwidth for solving  these issues, perhaps they could at least maintain one central Whitelist for common senders like Google and Outlook that could be pushed out during SM updates.  That would benefit the entire community. 
 
Make it available only to those with a valid support contract, so SM can still get paid for these efforts.
</rant>
 
6
CCC, good suggestions. I asked for something similar here (need votes):
 
Hopefully SmarterMail can make a few tweaks to the SMTP In to help out the admins and give users a better experience with quicker delivery.
0
Some type of improvement is definitely needed. Good suggestions.
4
Thank you everyone for the feedback and suggestions. I've created an internal ticket for this community post that I'd like to bring up with our SmarterMail developers. 
 
I'll be meeting with them on Wednesday to discuss further. I'll report back with more information from our discussion and hopefully we can get some of these suggestions implemented. 
Von See
Technical Support Supervisor
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
4
Von, sounds good. I'm sure the developers will say they're busy with SM16, but I think I speak for the community when I say:
It would be nice see a few incremental improvements to SM 15. We've all been renewing our licenses like good little customers, but haven't seen any real enhancements since March. And I'm guessing SM16 will not be out the door by the end of the year.
Thanks again, as we appreciate your efforts!
Kevin
0
CCC, I met with our developers this morning and we will be adding a feature request for automatically whitelisting larger providers based on SPF records.
Von See
Technical Support Supervisor
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
4
Greetings all, I met with the developers this morning. We will be adding a feature request for an option to automatically whitelist grey listing based on SPF Records for certain domains. 
Von See
Technical Support Supervisor
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
This will be a great help. Thanks!
0
Thank you sir for bringing this to the developers attention. This will be a big timesaver.
1
Von, you rock! and dance pretty good too (whd.usa :-) FWIW.

Maybe post an update on this thread also?
https://portal.smartertools.com/community/a86864/why-not-validate-trusted-senders.aspx
0
@ccc No problem at all.

@kevind haha, what can I say, I like to get down !

I've updated the other thread as well.
Von See
Technical Support Supervisor
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
@Von, if there is any chance that this same logic could be applied to a similar issue described on this thread (also related to SPF lookups and whitelisting)? If so, I can drop 99% of my whitelist entries which would make me dance too! :)

https://portal.smartertools.com/community/a87687/550-authentication-is-required-for-relay-enable-domains-smtp-auth-setting-for-local-deliveries.aspx
0
CCC, no problem at all. I've updated this thread and will bring this to our dev's on Friday.
Von See
Technical Support Supervisor
SmarterTools Inc.
(877) 357-6278
www.smartertools.com
0
Was this feature released in version 15.5 Enterprise. Its my understanding that an SPF pass would eliminate greylisting delays. I'm looking for a way to dynamically populate IP greylisting filters based on SPF check.
6
Hello, just following up on this thread from 7 months ago.
 
Was this SPF feature added to the 15.x series? Is it in the v16 release? Could really use it, especially for O365 servers.
 
Thanks!
1
Saw this problem again today with Amazon SES ( https://aws.amazon.com/ses/ ) -- 2FA messages from the bank would be delayed for hours.

Wondering if we could get an update. Thanks.
4
Hello!
 
Is there any update on this? Is there any plan for another method of exempting a whole SMTP cluster from Greylisting other than by manually adding lists of IPs?
 
We`re recently dealing with some clients that use a mailing system from OVH.net and their servers are really bad at handling Greylisting - if they receive the try again later message, their mailing system will cycle through dozens of gateways like 1.mo1.mail-out.ovh.net, 9.mo1.mail-out.ovh.net, 2.mo1.mail-out.ovh.net, 15.mo1.mail-out.ovh.net, 3.mo167.mail-out.ovh.net and so on and so on, each time with increase lag between tries. This causes huge delays in email delivery for clients that use OVH. We`ve tried whitelisting the IPs that we see in our SMTP logs, but almost each time they use another server and they have lots of them. We`ve also not been able to find any IP lists or IP ranges that they use, so this is getting out of hand.
 
Implementing a check based on SPF or something like that would be a business and life-saver - this could be seen as a basic feature that should go with Greylisting.
 
Looking forward to any updates on this.
5
How about adding a few options to greylisting?
  • Ignore greylisting if SPF passes
  • Ignore greylisting if RDNS passes
Or even simply do ignore greylisting if SPF passes. This would stop nearly all issues with delays and only apply greylisting under scenarios where someone is spoofing a domain etc.

Christopher

0
Adding 1000000000000000 ip addresses is simply not an option, especially for big carriers that can change constantly or move away from providers to new providers, then you have to hope you have maintained the correct list of IPs etc.

We have a tool that does this now, but it is a pain-in-the-arse to manage.

Christopher

0
So for "certain domains" - What qualifies a "certain domain"?

Christopher

0
+ 1 for me!!!!
1
Agreed, that's what I asked for in this thread:
https://portal.smartertools.com/community/a86864/why-not-validate-trusted-senders.aspx

Allow domains to skip greylisting (and other things like SpamAssassin) if SPF, DKIM, or some other quick test shows it as being authentic.

5
Here is a good idea that has a lot of votes, It took a year for SM to even consider it. It's over a year later and it's still "Under Consideration". Earth to SM are you going to take any action on this?
Kendra Support
http://www.kendra.com
support@kendra.com
425-397-7911
Junk Email filtered ISP
0
↑ THIS!!
0
Where is this feature in SM 16? I can't seem to find any info about it in the help file. I still have a ton of allowed IPs because of issues with Office 365 servers. Can I safely remove those IPs now or so I need to adjust this setting first?

Reply to Thread