Delay unrecognized domains - "Enable spool proc folder" - anti-spam idea?
Question asked by Colin M - April 22, 2015 at 10:36 AM
I've noticed a pattern of spammers that has been working rather well lately and that is sending spam from new domains before they can be blacklisted. I will often receive spam that doesn't hit any blacklists and then by the time I go check the blacklists manually the domain is listed! I can confirm by the timestamps that the email was received before the domain was blacklisted.
So, what I'd like to do is write a program that works similar to greylisting but only locally. That is, for any new domains (e.g. first instance seen within last 15 minutes), delay the email by say 15 minutes to give time for the domain to be blacklisted *before* applying any spam checks.
I see that SmarterMail has a "Enable spool proc folder" feature but it is not clear how this works. Does this spool proc folder feature apply before or after other filters? E.g. I have a remote spamassassin server that does most of my spam filtering so I would like for the message to be checked by spamassassin *after* I move it back into the spool. Is this how it works or is there any other way to accomplish this?
Note, I have greylisting enabled already. I'm not positive but I think the spammers are bypassing it by sending multiple duplicate emails in quick succession since often these spam messages come in duplicates.

4 Replies

Reply to Thread
Scarab Replied
April 22, 2015 at 12:05 PM
Smartermail Antispam checks are done prior to the Incoming message being moved to the Spool Proc folder. The reason these Antispam checks are done first and foremost is so that Smartermail can determine whether to block the message if it exceeds your Incoming Weight Threshold for those Spam checks that have Enabled for Incoming SMTP Blocking. After Antispam checks it then moves the message to the Spool Proc folder for additional processing, such as Declude or MailSniffer, or other third-party applications.
To be honest I can't recall whether Incoming messages are sent to the external SpamAssassin server before they are moved to the Spool Proc folder or after they are returned to the primary Spool. I'll have to double-check. However, if it is after then you can disable the Antispam check in Smartermail and enable that Antispam check in your external SpamAssassin server, in which case you could enable your Spool Proc folder, have a Scheduled Task to fire a script at 1 minute intervals that moves messages older than XX minutes back to the Spool, and have your external SpamAssassin server do the Antispam checks.
Colin M Replied
April 22, 2015 at 8:44 PM
I've implemented the "local greylisting" feature as described. All email for unrecognized domains is delayed for 15 minutes before the email is processed using the "Enable spool proc folder" option. Email that should not be delayed is only delayed by 100ms (to allow for possible filesystem delays).
The program is implemented in Node.JS and uses the fs.watch API to immediately detect new files (rather than polling). A local persistent database is kept to track the domains. Due to Node.JS event-based nature it is extremely resource-efficient. The script can install itself as a service or you can run it as any other node process for testing and errors are logged to a logfile. All existing files are processed when the service is restarted.
Dave Lerner Replied
March 27, 2016 at 10:21 AM
Hi Colin,
I'm still using this Node.js scheme and its working perfectly. However, I also have a second mail server, a free smartermail one configured as a backup MX, er, actually, its now configured for domain forward in SM gateway mode. In any case, recently, spammers have discovered this second server. I have the antispam setup the same way as on the primary, without bayesian of course. But without the spam delayer code running, it is subject to the same issues you describe above.
So, I set the whole thing up over on that server and couldn't understand why it was simply moving all the messages back into the queue. The answer was that on the backup mx box the SM configuration has not users or domains, so the .HDR files all have the last line as:
containsLocalDeliveries: False
Which causes a failure in our code here:
            if (lines.length > 5 &&
                lines[1].match(/@/) &&
                lines.some(function(line){ return line == 'containsLocalDeliveries: True'; })
....which in turn causes the code to write the files immediately back into the spool.
I modified that bit so it looks like:
            if (lines.length > 5 &&
                lines[1].match(/@/) &&
                lines.some(function(line){ return line == 'containsLocalDeliveries:'; })
..and it works fine.
Just curious about one other thing...did you ever figure out the order of processing in SM? I mean, @Scarab says the Antispam (and presumably the RBL checks) are done prior to moving the message into the proc folder. If that's the case, why would delaying the mail make any difference? How would you force SM to "redo" the spam checks if this is the case?
keith dovale Replied
July 1, 2016 at 3:43 AM
we use declude, which gets the mails from the proc folder, I would like to use this but I see it will most likey create an issue, as declude and the delayer will try grab the files from the proc folder, is there a way to maybe get SM to deliver to a folder lie "delay" then the delay prog scans them and then once the peiod is over return them to the proc folder where declude can run its own checks and return them to the spool folder ?
it would be nice to be able to use both

Reply to Thread