NginX proxy to a jailed Node.js app

khine · May 23, 2014

I am trying to run a couple of FreeBSD jails for my web server and application server, which is running Node.js. I have one network card (igb0) with a custom loopback interface (lo666) for which I have created 3 aliases, here is part of my /etc/rc.conf.

Code:

   dumpdev="AUTO"
    zfs_enable="YES"
    sshd_enable="YES"
    local_unbound_enable=yes
    ifconfig_igb0="inet 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255"
    # Custom loopback interface
    cloned_interfaces="lo666"
    ifconfig_lo666_alias0="inet 10.6.6.6 netmask 255.255.255.255"
    ifconfig_lo666_alias1="inet 10.6.6.7 netmask 255.255.255.255"
    ifconfig_lo666_alias2="inet 10.6.6.8 netmask 255.255.255.255"
    # Default router
    defaultrouter="192.168.1.254"

on my /etc/pf.conf I have this:

Code:

    ### Interfaces ###
     ExtIf ="igb0"
     IntIf ="lo666"
    
    ### Hosts ###
     IP_PUB ="192.168.1.1"
     IP_JAIL = "{10.6.6.6, 10.6.6.7, 10.6.6.8}"
     IP_JAIL_WWW = "10.6.6.6"
     IP_JAIL_DBS = "10.6.6.7"
     IP_JAIL_APP = "10.6.6.8"
     NET_JAIL="10.6.6.0/24"
     ### Ports ###
     PORT_WWW="{80,443}"
     PORT_NODE="{1337,8080}"
    
    scrub in all
    
    # nat all jail traffic
    nat pass on $ExtIf from $NET_JAIL to any -> $IP_PUB
    # WWW
    rdr pass on $ExtIf proto tcp from any to $IP_PUB port $PORT_WWW -> $IP_JAIL_WWW
    rdr pass on $IntIf proto tcp from any to $IP_JAIL_WWW port $PORT_NODE -> $IP_JAIL_APP

So running

Code:

    # pfctl -sn
    nat pass on igb0 inet from 10.6.6.0/24 to any -> 192.168.1.1
    rdr pass on igb0 inet proto tcp from any to 192.168.1.1 port = http -> 10.6.6.6
    rdr pass on igb0 inet proto tcp from any to 192.168.1.1 port = https -> 10.6.6.6
    rdr pass on lo666 inet proto tcp from any to 10.6.6.6 port = 1337 -> 10.6.6.8
    rdr pass on lo666 inet proto tcp from any to 10.6.6.6 port = 8080 -> 10.6.6.8

So from the the WWW jail I can

Code:

root@www:/ # curl http://10.6.6.8:1337
    Hello World

And also from the APP jail I can see the nginx home page

Code:

root@app:/# curl http://10.6.6.6
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    ...
    </html>

So, why do I get a 502 Bad Gateway, error when I try to access this through the browser? Here is my nginx.conf.

Code:

    server {
          server_name  web.domain.tld;
            location / {
                    # For Read Requests
                    proxy_pass http://10.6.6.8:1337;
                    proxy_set_header        Host            $host;
                    proxy_set_header        X-Real-IP       $remote_addr;
                    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            }
        }

Maybe I am overcomplicating it and what I wanted to achieve is to block port 1337 to external users.

Any advice much appreciated.

SirDice · May 23, 2014

khine said:
Maybe I am overcomplicating it and what I wanted to achieve is to block port 1337 to external users.

There's no redirection rule on the external interface to forward port 1337 so external users don't have access to it.

storvi_net · May 23, 2014

So what exactly do you want to achieve?

Using the nginx as a reverse proxy for 10.6.6.8 for all users? Then you can delete the last two rules, because all requests for any webpage are redirected to the proxy-jail, which chooses the correct server based on the "ServerName".
Because of the same network, there is no "routing" and "filtering". I have the same ruleset with one NAT for internet access, one rule for redirecting the web traffic (and another for HTTPS).

Regards
Markus

NginX proxy to a jailed Node.js app

khine

SirDice

Administrator

storvi_net