4
SmarterMail Linux Beta with nginx
Question asked by Patrick Jeski - 5/24/2024 at 6:35 AM
Unanswered
I have SmarterMail running with multiple domains using nginx. After SmartTools helped me resolve the status indicator spinner issue, I decided it might be helpful to some of you to see what I've done that works, and hopefully some of you that have more experience with linux and nginx can point out mistakes or better practices to me. My work is based on the sample SmarterTools provided and various samples on the web.

This is an Ubuntu 24.04 server install. I am not going to detail all the steps to install nginx, there's plenty on the internet for that. I created the directories for the certificates shown in my site file below, and I am using Certify the Web on a windows machine with a post-deployment task to sftp the certificates to those directories. You can't use a .pfx with nginx, so you'll want the chain (primary certificate + intermediate) in pem format. I intend to get certbot working, but since I already have a functioning CTW config, I went with that for now.

I created the site file smartermail in the /etc/nginx/sites-available directory, containing:
server {
    listen 80 default_server;
    server_name _;
    return 301 https://$host$request_uri; #permanent redirect to https
}

server {
    listen 443 ssl http2;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; #HSTS - Start with a shorter value and make sure everything works first
    server_name mail1.domain1.com;
    ssl_certificate /etc/nginx/ssl/certs/domain1.com.chain; #point to your actual certs
    ssl_certificate_key /etc/nginx/ssl/private/domain1.com.key;

    location / {
        proxy_pass http://localhost:17017;
        proxy_redirect http://localhost:17017 https://mail1.domain1.com; #I don't think this line is needed, doesn't seem to do anything
    }
}

server {
    listen 443 ssl http2;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    server_name webmail.domain2.com www.webmail.domain2.com;
    ssl_certificate /etc/nginx/ssl/certs/domain2.com.chain;
    ssl_certificate_key /etc/nginx/ssl/private/domain2.com.key;

    location / {
        proxy_pass http://localhost:17017;
        proxy_redirect http://localhost:17017 https://$host$request_uri; #I don't think this line is needed
    }
}

server {
    listen 443 ssl http2;
    #add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    server_name webmail.domain3.com www.webmail.domain3.com;
    ssl_certificate /etc/nginx/ssl/certs/domain3.com.chain;
    ssl_certificate_key /etc/nginx/ssl/private/domain3.com.key;

    location / {
        proxy_pass http://localhost:17017;
        proxy_redirect http://localhost:17017 https://$host$request_uri; #I don't think this line is needed
    }
}

Be sure to substitute your own domain names and cert names.

Then create a symbolic link to that file in /etc/nginx/sites-enabled (and while you're there, delete the symlink in sites-enabled for default).

Next I created smartermail.conf in /etc/nginx/conf.d containing:
# Additional configuration for SmarterMail
    ssl_session_cache shared:SSL:50m;
    ssl_session_timeout 5m;

    # Proxy settings
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $host;

    # Buffer settings
    proxy_buffer_size 4096k;
    proxy_buffers 8 4096k;
    proxy_busy_buffers_size 4096k;

    # Timeouts
    proxy_connect_timeout 1200s;
    proxy_send_timeout 1200s;
    proxy_read_timeout 1200s;

    # SSL Offloading
    proxy_ssl_server_name on;
    proxy_ssl_protocols TLSv1.2 TLSv1.3;

Note that the default nginx.conf in /etc/nginx contains:
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;

Then, test the configuration:
sudo nginx -t

And restart nginx:
sudo systemctl restart nginx

Please feel free to comment, I'd appreciate the help.



23 Replies

Reply to Thread
0
k Replied
hi, can you saw client's real ip address of webmail? 
I can only saw nginx's LAN ip. 
have setup your example, but I'm using docker version, everything is fine except this one.
1
Matt Petty Replied
Employee Post
Add these to your NGINX config and it should fix that.


        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;

I'd recommend setting these too, since some of the web protocols do long-polling.

        # Buffer settings
        proxy_buffer_size 4096k;
        proxy_buffers 8 4096k;
        proxy_busy_buffers_size 4096k;

        # Timeouts
        proxy_connect_timeout 1200s;
        proxy_send_timeout 1200s;
        proxy_read_timeout 1200s;


Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com
0
k Replied
thanks Matt, after changed nginx config, have reload/restart nginx. and restart/re-create smartermail container. still seeing nginx's lan ip in smartermail's admin panel.

nginx version: nginx/1.26.1
smartermail container version: smartertools/smartermail:100.0.8979

part of nginx conf ( proxy_pass to is incomplete due to I can post URL when submitting)
        # Buffer settings
        proxy_buffer_size 4096k;
        proxy_buffers 8 4096k;
        proxy_busy_buffers_size 4096k;

        # Timeouts
        proxy_connect_timeout 1200s;
        proxy_send_timeout 1200s;
        proxy_read_timeout 1200s;


 location ^~ / {
      proxy_http_version 1.1;
      proxy_set_header "Connection" "";

      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Host $host;
      proxy_pass http.....;
}
0
Patrick Jeski Replied
Are you sure it's an nginx issue and not a docker issue?
0
Matt Petty Replied
Employee Post
Docker shouldn't be using NGINX, we actually use Kestrel (dotnet's built in web server) for docker which allows us to process the connections directly and control how we serve the certificates. I checked our docker server here on one of the domains we host and it looks to be showing the external web IP's correctly in the administrative log.

If you have NGINX setup in front of the docker, then you would wanna add those things to the config. If your still having issues with this scenario this could potentially be something in our kestrel config we need to do to allow those IP's to come through.
Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com
0
k Replied
according to a post on stackoverflow, when use kestrel with reverse proxy, need 1 more line of code in back-end: 
const ipAddress = req.headers['x-forwarded-for'] as string;
source:  stackoverflow.com/a/64388115
0
k Replied
and the reason of willing to put sm after reverse proxy, I found m$ sum it well : 

A reverse proxy:
  • Can limit the exposed public surface area of the apps that it hosts.
  • Provides an additional layer of configuration and defense.
  • Might integrate better with existing infrastructure.
  • Simplifies load balancing and secure communication (HTTPS) configuration.

for example, with nginx I can easily block access to these path, I found they can be access without authorization. are there other path like these?

  1. /Documentation
    like /Documentation/api#/reference/SmarterMail.Web.Api.ChatController
  2. /api
    like /api/v1/auth/login-settings
  3. /About
    like /About/checkup
4
John C. Reid Replied
All I can interject is that those long and confusing configs you are using with Nginx and the entire need to run an Ajax AutoSSL infrastructure would just disappear and you would only need a single line in a Caddy file per reverse proxy domain if you used Caddy instead. 

The hardest thing to learn about Caddy is to not overthink it, it is simpler than you think and in most cases the defaults are what you want. Meaning that if you can't come up with what to put in the config, it is probably not needed as the only things to go into the config are overrides for the defaults or terms that cannot be a default.
0
Tan Replied
Hmmm im keen to follow up on this as i plan to setup nginx too as web server too.
0
k Replied
I have open a ticket about this months ago, support staff advise me to use IIS instead, or put IIS between nginx and smartermail.
0
Jorel Haggard Replied
Employee Post
K T,

In my understanding, that's essentially how it works with Linux too. Kestrel web server is what Smartermail uses by default (analogous to IIS on Windows), so you can either use Kestrel on its own or use it between Smartermail and Nginx. 

Best regards,
Jorel Haggard System/Network Administrator SmarterTools Inc. www.smartertools.com
0
k Replied
Hi Jorel,
in recent version of smartermail, iis is doing reverse proxy only, application is served by kestrel.
with IIS, smartermail can get client's real ip address, with nginx is not.
0
k Replied
try to use https with kestrel directly (docker version 9014, without reverse proxy)
1. installed valid pfx (confirm the pfx is ok/valid in windows)
2. enabled  (force all traffic via https/enable hsts) in 'General' tab, save settings, recreate/restart container.
3. try in cli and chrome, only HTTP works. 

seems i should go to windows version??

test with curl 
curl -vv -IL https://m2.xx.xx
*   Trying 172.30.100.3:443...
* Connected to m2.xx.xx (172.30.100.3) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
* error:0A000126:SSL routines::unexpected eof while reading
* Closing connection 0
curl: (35) error:0A000126:SSL routines::unexpected eof while reading
~$ nc -vz m2.xx.xx 443
Connection to m2.xx.xx (172.30.100.3) 443 port [tcp/https] succeeded!
0
Jorel Haggard Replied
Employee Post
K, 

Could you start a support ticket with these details so i can dive into it for you? 

Many thanks,
Jorel Haggard System/Network Administrator SmarterTools Inc. www.smartertools.com
0
k Replied
Hi Jorel,

no offence, but I guess at sm side there is no "complete" linux test environment?  
2
John C. Reid Replied
@k,

On the contrary, I know at least 2 of the techs that have a Linux server setup for their own testing, and I would be very surprised if most all if not all do. If the techs have it setup, I can almost guarantee the devs have it setup.

Just because it is still the same .NET 8 base does not mean they avoid keeping both environments for testing. There are always going to be cases of things different between the two installs simply because Windows Server and Linux are so fundamentally different.
John C. Reid / Technology Director John@prime42.net / (530) 691-0042 1300 West Street, Suite 206, Redding, CA 96001
0
Matt Petty Replied
Employee Post
A lot of my stuff I'm doing right now during development is tested via Docker, the team has various preferences while developing someone else uses WSL2, and others are just doing normal dotnet+iisexpress combos. We also moved smartertools.com to ubuntu a couple months ago, and we've got a real domain setup on Docker and another domain on a Windows server. We did this so that we technically have production servers of each type, which smartertools.com being the biggest on Ubuntu.
We also have numerous test servers, ranging from "long standing" test servers that we use for partners I think that runs on Ubuntu maybe Windows, to different combinations, we have some special language windows servers as well. 
Matt Petty Senior Software Developer SmarterTools Inc. www.smartertools.com
0
k Replied
I am going back to eval windows version again, if upstream has no nginx+docker/linux environment, I am at risk.
3
John C. Reid Replied
Just because they don't provide a reverse proxy in the install doesn't mean you can't use one. I have Caddy web server on a separate machine in front of mine, and my entire mail server has no public IPs. The only access is via one of reverse proxy for web with Caddy handling the SSL Certs, port forwarding for the mail protocols (on my production Windows install it is 1:1 NAT), and Apache Guacamole for SSH and RDP access. Don't blame the developers for your inability to secure the machine. It doesn't matter what software you use, that is on you as the system admin.
John C. Reid / Technology Director John@prime42.net / (530) 691-0042 1300 West Street, Suite 206, Redding, CA 96001
0
k Replied
@John, in your setup, can sm get client's real ip address? (instead of canddy's LAN ip)

ps, when did I blame anyone? 

thank you.
0
k Replied
@Jorel, actually I have submitted a ticket before, and your colleague have remote to my side for diagnosis. 

the ticket id is [304-2DD99C59-0B3C]

thank you for your time :) 
1
John C. Reid Replied
@k
Yes, Caddy supports X-Forwarded-For, which passes the original IP. On the mail protocol, 1:1 NAT doesn't obscure the original IP. 

Also, you stated: "if upstream has no nginx+docker/linux environment, I am at risk."
Which implies that this is within the scope of this project and not including it is a security issue. It is neither within scope, nor it is a security issue. That can be seen as blame.

John C. Reid / Technology Director John@prime42.net / (530) 691-0042 1300 West Street, Suite 206, Redding, CA 96001
0
k Replied
Thank you for sharing your perspective. I didn’t imply blame or overstate the situation, just pointed out a relevant consideration from my side. I’d prefer we not dwell on this further.

Reply to Thread