Solved FreeBSD VM guest's jail can't be resolved from host

unknownuser

unknownuser

Hello everyone.

I will try to explain my setup as clear as I can. I have a Windows 10 Workstation where I run a FreeBSD VM on VMWare 12.

This FreeBSD VM is meant to run multiple Apache/PHP/Wordpress instances on multiple jails.

The FreeBSD guest is bridged through the HOST Ethernet NIC, where the HOST has the IP 192.168.1.130 and the FreeBSD guest has the IP 192.168.1.100. So far all good.

The jail setup at the other hand is on subnet 10.1.0.0/24 with jail IP 10.1.0.1 and is an alias of the em0 interface to lo1 as virtual interface.

From within the jail I can ping the Windows host just fine, gateway and the whole 192.168.1.0/24 LAN network.

The problem I have is when I try to ping the FreeBSD jail from the HOST or from any devices connected to the 192.168.1.0/24 LAN.

My understanding is that I need to add a static route in order to have the FreeBSD jail resolved from the host, however I have been hammering my head badly for the past 4 hours since all I tried did not work.

Here's my FreeBSD (host) setup.

/etc/rc.conf
Code: 
ifconfig_em0="inet 192.168.1.100 netmask 255.255.255.0"
cloned_interfaces="lo1"
ifconfig_lo1_alias0="inet 10.1.0.1 netmask 255.255.255.0"
gateway_enable="YES"
pf_enable="YES"
defaultrouter="192.168.1.1"

/etc/sysctl.conf
Code: 
security.jail.sysvipc_allowed=1
security.jail.allow_raw_sockets=1
net.inet.ip.forwarding=1

pf.conf
Code: 
ext_if="em0"
jail_if="lo1"

IP_PUB="192.168.1.100"
IP_JAIL_bsdsrv1="10.1.0.1"

NET_JAIL="10.1.0.0/24"

scrub in all

# nat all jail traffic
nat pass on $ext_if from $NET_JAIL to any -> $IP_PUB

# allow ICMP ping
pass inet proto icmp from any to any

# passing all traffic
pass out
pass in

Additionally the gateway is a OpenWRT router.

I am happy to provide further information if needed. Any help will be gladly appreciated.

Regards
 
Quick update. Adding the following static route in FreeBSD

Code: 
# route add -net 10.1.0.0/24 192.168.1.1
add net 10.1.0.0: gateway 192.168.1.1
where 192.168.1.1 is the LAN gateway; jail gateway is FreeBSD IP which is 192.168.1.100.

Pinging from the HOST I get Request timed out.

Some WireShark fun here:

E3mqASu.png


I also tried giving 192.168.1.100 (FreeBSD IP and jail's gateway) as gateway in the static route but the scenario does not change.

Please advise.
 
You're NAT'ing the traffic, so your jail isn't reachable directly. For the same reason a host behind a NAT router cannot be reached directly from the internet.
 
You can give up the NAT but then your gateway (the device at 192.168.1.1) needs to be told how to reach the 10.1.0.0/24 network by creating a static route on the gateway with 192.168.1.100 as the destination ("next hop") of the new route and of course 10.1.0.0/24 as the network part.

It may sound counter-intuitive that the static route has to be done that way but that is how routing works. You need to think about both forward and return paths of the traffic, not only the forward part.
 
kpa wow it partially worked, now I am able to reach the jail from the gateway and LAN wide 192.168.1.0/24.

As you suggested I added the following static route on the OpenWrt gateway.

Code: 
root@OpenWrt:~# ip route add 10.1.0.0/24 via 192.168.1.100
However at the other hand if I disable pf and I add the following static route in FreeBSD:
Code: 
# route add -net 10.1.0.0/24 192.168.1.1
the jail is unable to resolve dns's, but it does ping external IPs just fine. In resolv.conf I have both google DNS and the gateway IP since serve as a DNS server as well.

Also if disable pf and gateway and I reboot the system the jail starting process will hang at boot time till I kill it manually.

However, for as much as I would like to understand what's the mess with my setup, for me this is already a win :)

Thanks
 
Your second route on the FreeBSD system makes no sense because it already knows where the 10.1.0.0/24 network resides since the lo1 interface has an IP address from that subnet. Remove it and the traffic should flow both ways.

One additional thing. The OpenWRT should do outbound NAT for the 10.1.0.0/24 network, otherwise the jail won't be able to reach the internet.
 
Dang you are totally right, static route on freebsdFreeBSD was totally pointless, and yes thanks to your advice I have managed to forward packets from the openwrt gw to the jail by using this iptables rule in /etc/firewall.user

Code: 
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 10.1.0.0/24 -j MASQUERADE

On freebsdFreeBSD I disabled gw and pf and the jail is able to ping both external IP's and DNS's.

Thank you so much for helping me out whit this nightmare. As you can see my networking knowledges are awful, but if explained I can work my way out :)

Regards.
 
You must log in or register to reply here.
Back
Top