Multiple parts to this post:
- Fix IIS so that additional data is in its log files
- Fix IIS so that SmarterMail sees the correct IP address when external users go through a web proxy.
- Fix IIS so that external users cannot connect using objectionable user agents.
- Fix IIS so that external users cannot connect using objectionable protocols
- Syntax notes for IIS Rewrite Rules
All of this has been test and in production use for several weeks.
Part 1: Add columns to IIS log file
- Click on the IIS SmarterMail website object
- Click on the Logging icon
- Click on the [Select Fields] button
- Add these fields to the list:
- Name=x-forwarded-for, Type=Server Variable, Source=HTTP_X_FORWARDED_FOR
- Name=x-externalflag, Type=Server Variable, Source=HTTP_X_EXTERNALFLAG
- Name=x-proxiedip, Type=Server Variable, Source=HTTP_X_PROXIEDIP
- Name=remote-host, Type-Server Variable, Source=REMOTE_HOST
Normally, REMOTE_HOST and REMOTE_ADDR both contain the IP address. There is a system setting somewhere that can cause IIS to do a reverse IP lookup so that REMOTE_HOST becomes the ReverseDNS name. Of course, all of those DNS lookups will create overhead and DNS timeout delays, so I think its a bad idea. But it would have been better if I used REMOTE_ADDR everywhere that REMOTE_HOST appear in this discussion.
Part 2: Fix IIS so that SmarterMail sees the correct IP address when external users go through a web proxy.
Given:
- Internal users connect directly.
- External users connect through a web proxy that sets x-forwarded-for and forwards to SmarterMail.
- Only one web proxy is in the flow path.
Result for Internal user at 192.168.1.1:
- On message arrival to IIS, x-forwarded-for is not present.
- On message exit from IIS, x-forwarded-for contains "192.1681.1:12345"
- SmarterMail uses "192.168.1.1" as the source IP for filtering and logging.
Result for External user at 8.8.8.8 via web proxy at 10.10.10.10:
- On message arrival to IIS, x-forwarded for contains "8.8.8.8"
- On message exit from IIS, x-forwarded-for contains: "8.8.8.8,+10.10.10.10:12345"
- SmarterMail uses "10.10.10.10" as the source IP for filtering and logging
Problem: External users are anonymized.
Solution steps:
Disable the feature that appends data to X-forward-for.
- On the IIS server object, open the "Application Request Routing Cache" icon.
- On the right side, click "Server Proxy Settings"
- Change "Preserve client IP in the following header to an arbitrary name starting with "x-proxiedip" (The name is arbitrary. I think I had a problem with leaving it blank.)
- Note that this change applies to all websites, but SmarterMail should be the only production web site on your server.
Use IIS Rewrite rules to set X-Forwarded-For to a single value:
- On the IIS object for the SmarterMail website, open the URL Rewrite icon.
- Itemize the server variables that your rules will need to read or modify:
- click "View Server Variables"
- Add the variables you need. I am using:
- HTTP_USER_AGENT (Used to block unwanted client software)
- HTTP_X_EXTERNAL_FLAG (a variable I will populate)
- HTTP_X_Forwarded-For
- HTTP_X_PROXIED_IP
- Click "Back to Rules" when the list is complete.
- Add a rule that assumes an External source by default:
- Name = "External"
- Matches the pattern using regular expression with pattern .* and ignore case
- Conditions: None
- Server Variables: HTTP_X_EXTERNALFLAG value External
- Action: None
- Stop processing of subsequent rules: UNCHECKED
- Add a rule that corrects the variable for Internal connections:
- Name="Internal Direct"
- Matches the pattern using regular expression with pattern .* and ignore case
- Conditions: {HTTP_X_FORWARDED_FOR} does not match (.*+)
- Server variables:
- HTTP_X_Forwarded_For value {REMOTE_HOST} replace=CHECKED
- HTTP_X_EXTERNALFLAG value DIRECT replace = CHECKED
- Action: None
- Stop processing of subsequent rules: UNCHECKED
All of your rules must appear before the two provided by SmarterMail, because those rules cause subsequent processing to be stopped.
New result:
- If X-Forwarded-For is set on arrival to IIS, the value is preserved
- If X-Forwarded-For is empty on arrival to IIS, it is set to the IP of the client device.
- SmarterMail never sees a concatenated value in that field.
- SmarterMail always sees the intended client IP address.
Complications:
- If you have multiple web proxies in the path, the External rule will need to use a RegEx to strip the value of X-Forwarded-For down to the text string which precedes the first comma.
Part 3: Fix IIS so that external users cannot connect using objectionable user agents.
Goal: Block external user agent strings that represent a security risk.
The user-agent string is included in IIS logs by default, and we have added the x-externalflag at the beginning. Feed the logs into a parser program to find all of the user agents used internallly and externally. Use this to decide what user-agent strings to allow or block, and to develop the regex pattern that will accomplish what you want.
We have chosen to block:
- Outlook for Mobile, as a security risk because it goes through Microsoft Severs. I used an expression of {HTTP_USER_AGENT} matches "Outlook", ignore case
- Microsoft Office Outlook, since it downloads significant amounts of company data to personal devices. I used expression {HTTP_USER_AGENT} matches "^Microsoft\+Office", ignore case
- The RootEvidence bot, a network surveillance tool from cybersecurity company rootevidence.com. I used expression: {HTTP_USER_AGENT} matches "^RootEvidence", ignore case. I found this by examining my IIS logs.
I created three rules for these three scenarios. If the list of blocked useragents becomes long, it will make more sense to have one rule to detect the presence of an unwanted agent, and set a variable, followed by a second rule that applies a block if the source is external. But for the moment, I have three rules for three user agents. Here is on of them:
- Name = "Block RootEvidence bot"
- Matches the pattern using regular expression with pattern .* and ignore case
- Conditions (MATCH ALL)
- {HTTP_X_EXTERNALFLAG} matches EXTERNAL
- {HTTP_USER_AGENT} matches ^RootEvidence and ignore case
- Server Variable: None
- Action: Abort Request
Part 4: Fix IIS so that external users cannot connect using objectionable protocols
We want to block external connections to MAPI, for the same reason that we block Microsoft Office Outlook.
- Name="Block External MAPI
- Match the pattern using regular expression with pattern ^MAPI and ignore case
- Conditions
- {HTTP_X_EXTERNALFLAG} matches EXTERNAL
- Server Variables: None
- Action: Abort Request
Variations:
- To detect or block EWS, use pattern: ^EWS
- To detect or block WebMail, use pattern ^interface/root
- To detect or block ActiveSync, use pattern ^Microsoft-Server-ActiveSync
Part 5: Syntax notes
Request Headers vs Server Variables
- Request headers are automatically mapped to server variables. Anything not baked into IIS uses the following naming formula:
- The name is prefixed with "HTTP_"
- Hyphens become underscores
- Text is raised to uppercase.
- Example: The header "x-forwarded-for" becomes the variable "X-FORWARDED-FOR"
- Server variable must be added to the allowed server variables list before you can manipulate them with IIS rules.
- Mapping is bi-directional; the server variable is populated when the web request arrives, and the header is updated or added before the request is forwarded to SmarterMail.
- All custom headers should start with "x-"
- As indicated in my examples, when a server variable is referenced, it must be enclosed in braces. For example: {HTTP_X_EXTERNALFLAG} matches EXTERNAL
- When a variable is assigned a value, it is not delimited.
For example: HTTP_X_EXTERNALFLAG = EXTERNAL - RegEx results can be used to assign a value. {R:0}{C:0} is the first regex group returned by the most recent RegEx test. If you need this complexity, test carefully by sending results to a variable and adding the variable to the IIS log