Too Many Addresses in Internal Blocked Senders List Causes All Incoming Mail to be Deleted?

Problem reported by Jim Rosemary - 4/8/2017 at 8:46 PM

Submitted

(SM 15.5.6222) Here's a new one... A client called today to report that he stopped receiving all incoming mail. No one else in his domain and no one else on the server was reporting any such issue. He was using webmail, so it wasn't an issue with his settings (as it might have been had he been using Outlook or some other mail client). I checked by sending him a message and, sure enough, nothing arrived in his Inbox. So, I checked the logs and found my message to him: "Delivery for [me] to [client] has completed (Deleted) Filter: Internal Blocked Senders". All the other reported senders also showed the same "Internal Block Senders" log entry. So, I opened his Internal Blocked Senders list and searched. My address was not there and neither were any of the other senders he said could not get mail through to him. I checked the "userConfig.xml" file on the server and the addresses matched what was visible from the webmail interface. (I wanted to verify that there were not additional addresses in the xml file that just weren't visible from the webmail interface.) I then deleted all 6183 addresses in his Internal Blocked Senders lists (both "from specific address" and "from address" fields). A second test message was then able to get through to him.

Could this be an indication that there's some kind of limit to the Internal Blocked Senders list? Or, that when the list grows to a certain size. it simply blocks ALL incoming mail? Has anyone else experienced this issue?

Please double-check to make sure this isn't an issue in v16.

Jim Rosemary
New Tech Web, Inc.

14 Replies

Reply to Thread

Curtis Kropar www.HawaiianHope.org Replied

4/9/2017 at 8:54 PM

Do you still have the list available to search through ?

Are there any wildcards that were in there ?

www.HawaiianHope.org - Providing technology services to non profit organizations, low income families, homeless shelters, clean and sober houses and prisoner reentry programs. Since 2015, We have refurbished over 11,000 Computers !

Jim Rosemary Replied

4/9/2017 at 10:30 PM

Yes, we saved the lists offline. No, there are no wildcards in either list.

Jim Rosemary New Tech Web, Inc.

Employee Replied

4/10/2017 at 8:07 AM

Employee Post

Hi Jim. Do you happen to have a copy of this userConfig.xml file?

Jim Rosemary Replied

4/10/2017 at 10:54 AM

Rod, sorry, no. I didn't think to keep a copy of the userConfig.xml prior to deleting the Internal Blocked Senders.

Jim Rosemary New Tech Web, Inc.

Employee Replied

4/10/2017 at 12:20 PM

Employee Post

Hi Jim. No worries. I think I can test this without your XML file. I'll just create an IBS list with about 6,200 entries.

Scarab Replied

4/13/2017 at 12:41 AM

We have this occasionally happen to users when they flag a Spam message with the Block Sender function but there is a NULL value for the address. A NULL value in the Block Senders list has the same results as a wildcard.

Although then again, I don't think we've had any users with > 6000 addresses in their Block Sender list. The highest any of our users have is the @ 4800.

Jim Rosemary Replied

4/13/2017 at 1:10 AM

How would a null value for the address appear in the IBS list or the XML file? If it's just blank, how would that function as a wildcard?

Jim Rosemary New Tech Web, Inc.

Scarab Replied

4/13/2017 at 12:35 PM

The Null value entry looks like <> in the IBS. It gets added when an email that is flagged does not have a valid From: or Return-Path: which is common in Spam. By default this gets added to both the "From Address" and "Contains Specific Words & Phrases" sections. (I seem to recall empty quotes "" causes a similar issue) the "Contains Specific Words & Phrases" section is where this causes a problem.

This will cause any email that does not have a defined Return-Path:, Reply-To:, or From: field to be subject to the IBS rule, most commonly the Reply-To: is not frequently defined in legitimate email causing it to get flagged when that field empty if a Null value such as <> gets inadvertently added to the IBS.

Employee Replied

4/13/2017 at 12:37 PM

Employee Post

Hi Jim. I just wanted to follow up, as I've tested this locally. I created an Internal Blocked Senders list that contained 7,000 unique email addresses. First I tested email addresses that were at the bottom of this list to ensure that the entire IBS list was being searched, and those were blocked as expected. Then I sent emails from good email addresses, and they were indeed delivered to the inbox without issue. Unfortunately, I don't have a good answer as to what happened in your case. If you do experience this again, pull the userConfig.xml for the account, and then start a support ticket so we can see exactly what happened. I hope this helps.

Scarab Replied

5/2/2017 at 12:41 PM

Just had this happen today to a customer. In addition to the <> value the problem can occur with the following characters in an email address:

! (although an allowed character for UUCP Bang-Path & Hybrid Addresses under RFC-822, 2822 and 5322 RegEx treats a bang similar to a *)

: and :: (although an allowed character for Route-Addr Address and DEC/FidoNet Addresses under RFC-822, 2822 and 5322 RegEx treats a colon as a non-capturing group)

% (although "Ye Olde Arpanet Kludge" addresses have not been RFC compliant for over 30 years almost every MTA still supports them...not sure if Smartermail does or not)

I would assume that the other 12 RegEx reserved characters in an email address would result in similar symptoms:

(

)

[

]

Hope that helps narrow down searches when trying to find the needle in the Blocked Senders Filter haystack.

Which brings us to a quandary...perhaps SmarterMail should be sanitizing email addresses prior to adding them to the Blocked Senders Filter to prevent addresses containing special characters from filtering all incoming email. Maybe this should be turned into a request thread.

Rich Wheelwright Replied

5/4/2017 at 11:57 AM

I just had the same thing happen to me. had over 9000 blocked senders. Cut and pasted them to an excel spreadsheet and email started coming in however all email from the past 2 days is still missing. Any way to retrieve?

Scarab Replied

5/4/2017 at 3:55 PM

Rich,

If you have Mail Archiving enabled in SmarterMail you can restore any messages that were deleted by going to MANAGE > MESSAGE ARCHIVE SEARCH and entering the "Date Range" and "TO:" email address. Once those messages are found you would select ALL and click on [COPY TO MAILBOX] and enter the first part of the email address (i.e. example if the email address is example@domain.tld) in the "Username" field and usually Inbox in the "Folder" field to restore to.

Note: You have to have cleared the Blocked Sender that contained the invalid character prior to restoring any messages otherwise they will just get deleted again.

Rich Wheelwright Replied

5/4/2017 at 4:08 PM

Doesn't look like we have mail archiving enabled. What is weird is that if I go to reports and go to message traffic it says a lot of emails were delivered between today and 2 days ago when this happened but they aren't in my deleted items. How is It possible that emails came in but are just gone?

Scarab Replied

5/4/2017 at 4:37 PM

Rich,

As those messages are deleted by the Delivery Service rather than the Web Service they will not appear in the \Deleted Items folder. They are entirely gone from that account before they are even written to their Inbox. Without Mail Archiving they are sadly not recoverable.

If you have the Disk Space I strongly recommend enabling Mail Archiving for situations like these. We get about 3-4GB of email a day and our Mail Archive takes up only @ 750MB a day. Having it enabled saves a lot of headaches when users accidentally delete their Inbox, or were using POP3 and they experience a hardware crash on their PC taking all their emails with it, or in cases like these when a Blocked Sender gets added that contains a special reserved character.

If you only want to retain messages for x# of weeks/months/years you can have a script run as a nightly Scheduled Task to remove files from your Archive folder that exceed that time span as Smartermail never purges Archives.

Back to Community Threads

Please leave this box unchecked

14 Replies

Reply to Thread

Tags

Related Knowledge Base Articles