Nginx reverse proxy on host plus nginx in multiple host

V

Vitviggs

Hi all, looked around about this topic, found a lot of articles but all confusing. Appreciate if someone can make it clear. It is about jails with internals IP in which are running different websites(let say WP with each having its own database and own php and own nginx inside reach jails), on a freebsd vps with single public IP and nginx reverse proxy on host.

My question is:
1- What would be the nginx conf for the reverse proxy on host(there is no domain or website on host) to redirect secure and non-secure traffic to jails
2- what would be the nginx conf in each jail to serve local websites with secure traffic

3- what would be the pf conf for the above config


I managed to run only one jail with traffic redirected by pf but when comes in the second jail it doesn't work due to the port-IP unique bind. So we need a reverse proxy.

All the examples found around makes me confused cause they are not clear.
The scheme I want to use is Reverse proxy on host, no domain on host, websites inside jails with local servers each

Thanks for some example of configurations
 
While you could certainly use nginx for this purpose, I personally use net/haproxy for this. It's small, it's intended to be used as a reverse proxy and it has a lot of interesting features you can't do with nginx alone.

Vitviggs said:
3- what would be the pf conf for the above config
Click to expand...
You allow incoming connections to port 80 and 443 to the host. The connection is terminated there. A new connection is made from the host to any of the backend servers (i.e. your jails). There's no need for a redirect.
 
Terminate HTTP and HTTPS traffic with nginx@host.
Inside the jails you should have any configured web-server, which required for your sites.
So you can have 1st jail with Apache, 2nd jail with nginx, 3rd jail with something else,
and you can have even some jail with static-files and serve these files using nginx@host.
Usually you don't need to do something with PF for the jails. But sometimes you may want to have a NAT for jails.

Example of my nginx config for environment like your's. Tune it and use it as you want.
For case with few HTTP websites hosted in the one jail you can add another sitename to an appropriate nginx's section.

/usr/local/etc/nginx/nginx.conf
Code: 
load_module /usr/local/libexec/nginx/ngx_stream_module.so;
user  www www;
worker_processes  1;
error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;
timer_resolution 100ms;
worker_priority -5;
events { worker_connections 2048; }
stream {
        log_format basic '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time';
        access_log  /var/log/nginx/stream_access.log basic;
        upstream ssh {
                server X.X.X.X:22;
                        }
        server {
                listen          Y.Y.Y.Y:22;
                proxy_pass      ssh;
                }
        }

http {
    include       mime.types;
    default_type  application/octet-stream;
     log_format myssl '$remote_addr [$time_local] $host "$request" $status  "$http_referer" "$http_user_agent" "$ssl_protocol" "$ssl_cipher" "$ssl_server_name" "$ssl_session_reused"  ';
access_log /var/log/nginx/nginx-access-myssl.log myssl;
   server_tokens off;
    sendfile        on;
    keepalive_timeout  65;
    proxy_connect_timeout 20;
server_names_hash_max_size 2048;
server_names_hash_bucket_size 64;
        set_real_ip_from  x.x.x.x;
        real_ip_header     X-Real-IP;
        real_ip_recursive on;
include "/usr/local/etc/nginx/servers.conf";
}

/usr/local/etc/nginx/servers.conf
Code: 
server {
    listen X.X.X.X:443 ssl;
    server_name online.site www.online.site;
    ssl_certificate /usr/local/etc/nginx/ssl/online.site/online.site.cer;
    ssl_certificate_key /usr/local/etc/nginx/ssl/online.site/online.site.key;
    ssl_ciphers EECDH:+AES256:-3DES:RSA+AES:RSA+3DES:!NULL:!RC4;

ssl_session_cache shared:SSL:20m;
ssl_session_timeout 180m;

#disable SSLv3 (poodle)
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    client_max_body_size 20m;
        location / {
                proxy_pass http://192.168.46.4:80;
                proxy_redirect off;
                proxy_set_header Host           $ssl_server_name;
                proxy_set_header X-Real-IP            $remote_addr;
                proxy_set_header X-Forwarded-Proto      $scheme;
                   }
        location ~ /\.ht { deny  all; }
      }
####
    server {
        listen       X.X.X.X:80;
        server_name     m.x.com ;
        client_max_body_size 20m;
        location / {
                proxy_pass http://192.168.46.5:80;
                proxy_redirect off;
                proxy_set_header Host                 $host;
                proxy_set_header X-Real-IP            $remote_addr;
                   }
      location ~ /\.ht { deny  all; }
           }
####
    server {
        listen       X.X.X.X:80;
        server_name  s.com;
        location / {
                root /home/jails/wwwold/home/defaultsite;
                expires 1m;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html { root   /usr/local/www/nginx-dist; }
        location ~ /\.ht { deny  all; }
    }
 
Last edited:
im
Thanks for that. Allow me a bit discussion on it. More xactly on servers.conf. So, the point is that I want that the websites inside the jails to be accessed on https. In this case, the local nginx in each jails, should be certificated with certbot? Or is enough to certificate only the proxy on host?
Then I see in the servers,conf at the first server directive that we use server.online as server_name example. To what exactly these domains refers as I have in jail1 the domain site1.com and in jail2 the domain name site2.com and no domain on host?
when I run certbot on host for the proxy , certbot will ask me for which domain to ask the certificates? what domain should I input? the site1.com and site2.com from jail1 and jail2? And should I create manually the directories for the keys(/usr/local/etc/letsencrypt/live/...)?
Thanks for some more info on these little extra details
 
Vitviggs said:
In this case, the local nginx in each jails, should be certificated with certbot
Click to expand...
You can have certificates only on nginx@host.
It is no reason to have additional certificates inside of each jail.
Nginx@host can communicate with webservers@jails over HTTP.
For external clients HTTP+HTTPS service will be provided via nginx@host.

Vitviggs said:
Or is enough to certificate only the proxy on host?
Click to expand...
yes, certs only on the host. It works for me.

Vitviggs said:
Then I see in the servers,conf at the first server directive that we use server.online as server_name example. To what exactly these domains refers as I have in jail1 the domain site1.com and in jail2 the domain name site2.com and no domain on host?
Click to expand...
Nginx should to know the domains to serve.
Each "server" directive describes a domain (or a list of the domains) to serve.
There are an IP address to listen on the host, a location of the site_root (local or remote), etc.
You should specify for nginx every domain name for which you want to have access.
It is no corellation between hostname of FreeBSD host and domains specified for nginx.

Vitviggs said:
when I run certbot on host for the proxy , certbot will ask me for which domain to ask the certificates? what domain should I input? the site1.com and site2.com from jail1 and jail2?
Click to expand...
You should to input the real domain names which you want to access via web-browser.
Usually I stop the nginx@host before run certbot.

Vitviggs said:
And should I create manually the directories for the keys
Click to expand...
In my opinion, you should not to do it. But you can configure everything as you want.
In my opition, the better way is to use default path for the keys, and specify a correct path for the keys in nginx configs.

Also, you should to know, that nginx will use 1st described HTTP and HTTPS server as a default server for unspecified domains.
So you should to have configured HTTP default server with a stub site configuration.

The same situation for 1st HTTPS server (or for default server). nginx will push certificates from default HTTPS server if someone will try to access via HTTPS to some site which have no configured HTTPS. It is possible to avoid this behaviour defining the stub HTTPS default server:
Code: 
server {
 listen x.x.x.x:443 ssl default_server;
 ssl_reject_handshake on;
}
 
Last edited:
im
Thanks, I managed to have it working in the end.
Anyway I set the reverse proxy inside another jail, redirect traffic to that jail(proxy) via pf, then from the proxy jail to other jails via nginx reverse proxy.
works fine for php driven websites,but I am struggling to have it working for other apps. For example I am trying to install a VPN server inside one jail, but not really succsesfull so far. Tried ocserv, by some reason the service doesn't start, tried softether, it installs ok but can't get it to connect to client.
I use Bastille jails, they are pretty light and easy to manage.

Thanks for the tip with ssl on reverse proxy only
 
You must log in or register to reply here.
Back
Top