Jails with Single Public IP (in VPS) cannot reach internet

Z

zennybsd

Hi:

Created jails with a single public IP in a VPS to separate the services. There are two problems. <public_static_IP> quoted below is the public static IP provided by VPS provider:

  • When I try to assign a static IP (by changing the
    Code: 
    ifconfig_em0="inet <public_static_IP>/24"
    in /etc/resolv.conf, the host VPS cannot reach internet (with the relevant nameservers in /etc/resolv.conf).

    However, when I change that to DHCP
    Code: 
    ifconfig_em0="DHCP"
    the machine connects to Internet with the same nameserver in /etc/resolv.conf.
  • With DHCP option enabled as above, the jails cannot reach the internet (tried with both ezjail and qjail).
My configurations are:

In /etc/rc.conf, I added for jails:
Code: 
ipv4_addrs_lo1="192.168.59.1-9/29"

In /etc/pf.conf:

Code: 
PUB_IP="<public_static_IP>"
WWW_JAIL="192.168.59.3"
NET_JAIL="192.168.59.0/24"
PORT_JAIL="{80,443,2022}"
scrub in all
nat pass on em0 from $NET_JAIL to any -> $PUB_IP
rdr pass on em0 proto tcp from any to $PUB_IP port $PORT_JAIL -> $WWW_JAIL

Although I have made necessary changes in /etc/ssh/sshd_config of the $WWW_JAIL and sshd daemon is started:

Code: 
Port 2022
ListenAddress 192.168.59.3
PermitRootLogin yes

The jails cannot reach the Internet not I can ssh to the jails using port 2022.

#pfctl -sn
Code: 
nat pass on re0 inet from 192.168.59.0/24 to any -> <public_static_IP>
rdr pass on re0 inet proto tcp from any to 192.168.53.40 port = http -> 192.168.59.3
rdr pass on re0 inet proto tcp from any to 192.168.53.40 port = https -> 192.168.59.3
rdr pass on re0 inet proto tcp from any to 192.168.53.40 port = down -> 192.168.59.3


Any suggestions to overcome this problem? Thanks!
 
zennybsd said:
In /etc/pf.conf:

Code: 
PUB_IP="<public_static_IP>"

nat pass on em0 from $NET_JAIL to any -> $PUB_IP
Click to expand...
Don't use things like this. It's not going to work if your IP address changes. Instead use something like this:
Code: 
nat pass on em0 from $NET_JAIL to any -> (em0)
 
@SirDice: thanks for the useful info. I made the changes accordingly.

Yet jails cannot reach the net!

From jails host:
Code: 
# time nc -z -w 2 www.freebsd.org 80
Connection to www.freebsd.org 80 port [tcp/http] succeeded!
0.000u 0.004s 0:00.20 0.0%	0+0k 0+0io 0pf+0w

But from jails:
Code: 
proto-www /root >time nc -z -w 2 www.freebsd.org 80
^C0.000u 0.004s 0:22.46 0.0%	0+0k 0+0io 0pf+0w

I could not figure out exactly what prevents the jails to access the net? Any inputs?
 
Last edited by a moderator:
Did you enable routing on the host? In /etc/rc.conf:
Code: 
gateway_enable="YES"

I also noticed you're using em0 in pf.conf but your pfctl -sn shows re0, make sure you use the correct interface. The best tool to debug issues like this is tcpdump(1). Run it on your external interface and make sure NAT works correctly. It's possible the host is forwarding packets but using the private IP address as a source. You'd never get a reply and this may explain the time-outs.
 
You must log in or register to reply here.
Back
Top