Seeing a trend I don't understand
Question asked by John C. Reid - 1/13/2017 at 11:23 AM
For the 3rd time now I have seen this and I need some help figuring out what might be going on.
Customer calls and they are angry. They have been having E-mail dropped for weeks and they are just figuring it out. Some people never get their E-mail, and it is not going into the Other person's spam. I say OK, I will look at it. Please give me the dates and to addresses you were sending to.
From here it starts to get a bit odd. My SMTP and Delivery logs are on detailed, always have been. Looking through the logs I can find zero evidence that they ever even attempted to send a message to the person they claim did not receive it. Searching the SMTP log for the MAIL From: <sender@domain.com> I see a pattern that leaves me scratching my head.
There are log entries that look normal. It is obvious that they have the E-mail client software setup correctly and using my SMTP server for outbound mail. They authenticate, and their client goes through the motions as you would expect ending with the "Data transfer succeeded, writing mail to xxxxxxxx.eml" Those messages make it to the recipient as expected.
Then I also see a lot of these, with the from address being the user that is having issues:
[2017.01.02] 19:18:21 [][9370777] rsp: 220 shastaemail.com Tue, 03 Jan 2017 03:18:21 +0000 UTC
[2017.01.02] 19:18:21 [][9370777] connected at 1/2/2017 7:18:21 PM
[2017.01.02] 19:18:22 [][9370777] cmd: EHLO 95-173-225-26.milleni.com.tr
[2017.01.02] 19:18:22 [][9370777] rsp: 250-shastaemail.com Hello []250-SIZE 52428800250-AUTH LOGIN CRAM-MD5250-STARTTLS250-8BITMIME250 OK
[2017.01.02] 19:18:22 [][9370777] cmd: MAIL From:<sonya@shasta.com>
[2017.01.02] 19:18:29 [][9370777] rsp: 554 Sending address not accepted due to spam filter
[2017.01.02] 19:18:29 [][9370777] Mail rejected due to SMTP Spam Blocking: Barracuda - BRBL, CBL - Abuse Seat - DO NOT USE FOR OUTGOING MAIL, MAILSPIKE Z, SORBS 06 - RECENT, SORBS 07 - WEB, SPAMHAUS - PBL 2, SPAMHAUS - XBL 1
[2017.01.02] 19:18:29 [][9370777] disconnected at 1/2/2017 7:18:29 PM
Notice that the connection is initiated by a remote host (typically a IP one should not trust as well) but it claims the  from is my user. It then get immediately rejected. There appears to be a correlation to the users that have E-mail mysteriously disappearing.
In summary, three things related to users having this issue:
  1. E-mail that disappears have nothing to indicate that it ever existed anywhere in my logs.
  2. E-mail that delivers looks perfectly normal and  would suggest they are properly using my SMTP server (I have a DMARC policy for reject set in DNS and we are signing DKIM, so they must use our SMTP server to pass DKIM. SPF is also in place, to the DMARC would still pass for that if the IP matches a listed server.)
  3. I have these odd, reverse initiated SMTP sessions claiming to be from the affected user.
John C. Reid  / Technology Director
John@prime42.net  / (530) 691-0042
1300 West Street, Suite 206, Redding, CA 96001

4 Replies

Reply to Thread
John C. Reid Replied
John C. Reid / Technology Director John@prime42.net / (530) 691-0042 1300 West Street, Suite 206, Redding, CA 96001
Jay Altemoos Replied
Just by looking at the above log you posted it appears that someone or something is impersonating that user. I did a reverse look up on that IP and it appears to be coming from Turkey. It's most likely a bot. On our server, I have a rule set up under Security -> Advanced Settings -> Abuse Detection specifically for SMTP sessions attempting bad username and password combinations.
What version of SmarterMail are you using?
As far as emails that disappeared, if it's not in the log then it didn't send through your server. It might still be stuck in their outbox if they are using an email client? I would start by double checking their mail settings to cover that aspect. The other possibility is if the user's machine is infected, that might also be the reason why you are seeing the suspicious connection IP above from Turkey. If they have a virus it is probably hijacking their mail client or trying to reroute their DNS. In which case, the user could lose emails they sent out if the latter is happening. Not saying that's a definite, but it's a start somewhere. Also, not sure how many people you host on your server, but I would think if it's a direct problem with SmarterMail itself you would be hearing lots of complaints from other users as well. If it's just this one, then my bet would be an infection if they are using a mail client.
My suggestions, 1.) Verify the settings on this user's mail client if they are using one and 2.) have the user scan their machine for spyware / virus. There's a bunch of free utilities that they can use to scan with, MalwareBytes, JRT, and Emsisoft Emergency scanner are ones that come to mind. Antivirus if they don't have one they could use AVG free or Avira Free.
Besides that, if they continue to have issues and you see a pattern with other users as well, I would open a ticket with support.
John C. Reid Replied
SmarterMail Version - 15.5.6222
Ok, so it looks like I need to clarify where what I stated was exact and I am not guessing. I understand trying to read behind the lines because of how odd this is, but that is also why I am here asking.
1) This is not a single client / e-mail address / user, it is multiple. So we are not looking at malware on a single computer.
2) When I say they sent it and there is no SMTP log, I mean exactly that. The person sends the message (all the test cases so far have been Outlook users) and the message goes out, gets copied from the outbox to sent, and there are no errors. I check the SMTP logs shortly thereafter and there is no indication of the connection at all.
3) The log snippets above are not attacks or impersonations. I get exactly 1 of these entries for each missing e-mail INSTEAD of the SMTP log entries I SHOULD HAVE SEEN when the customer sent the e-mail. It is a 1:1 relationship. The time and date stamp is between 30 seconds to 2 minutes delayed from where there should have been an entry for the SMTP connection from the E-mail client software.
4) It is not every E-mail they send.
5) I have verified the E-mail client software is setup to use (authenticated) this server for SMTP, and for most E-mail the evidence that the client is doing that is in the SMTP log. However, on the occasional instance when it is not I will instead get these delayed, reverse connections, the IP is always from some random IP oversees, but always a middle eastern or mediterranean country.
John C. Reid / Technology Director John@prime42.net / (530) 691-0042 1300 West Street, Suite 206, Redding, CA 96001
John C. Reid Replied
So it would seem this is at least as baffling to everyone else that reads this as it is to me? I guess that is what you get when looking for an explanation to the apparently impossible. For right now we are going to ignore this issue.
Possible this would make it a non-issue → https://portal.smartertools.com/community/a88697/reverse-proxy-appliance-to-offload-ids-and-av_.aspx I could still use some weighin here.
John C. Reid / Technology Director John@prime42.net / (530) 691-0042 1300 West Street, Suite 206, Redding, CA 96001

Reply to Thread