Solved Remote LAN-in-a-box?

C

ccammack

I would like to access a remote jail/VM host behind a family member's home router without making any changes to their router config or adding additional hardware. (I suppose they could treat the host as a guest to disallow LAN access and allow only outgoing traffic.)

I think I need these things:
  1. the host should get its default route and IP from whatever device it plugs into over DHCP
  2. pf on the host to protect it from the LAN and to NAT the outgoing traffic (allow only SSH from the LAN to the host for setup and emergencies)
  3. a new disconnected bridge on the host with its own private IP that is not bound to the physical interface
  4. create all of the jails and VMs on the private bridge network
  5. configure pf to route between the bridge and the physical interface
  6. wg(client) on the host to connect with wg(server) on a public VPS
  7. dnsmasq on the host, listening to the bridge, to serve DHCP to the jails/VMs and to serve DNS to both the jails/VMs and the host. dnsmasq will fall back to 9.9.9.9 for unknown domains
Does that sound like the right approach? I'm especially fuzzy on network design and routing with pf, so I'm looking for details and simplifications.
 
Last edited:
I do this with tailscale, which manages all the details including authorization w/o me having to worry about it too much.
I use this for accessing my internal network which is behind a T-Mobile wireless gizmo with no user serviceable network config.
I can also access this via my mac or windows box too, since there's tailscale clients for that, as well as my freebsd laptop.

I know this isn't directly helpful to the path you are on, but might be an alternative worth considering.
 
After some experimentation, it mostly works using this bridge setup:

Code: 
# /etc/rc.conf
cloned_interfaces="bridge0"
autobridge_interfaces="bridge0"
autobridge_bridge0="em0"
ifconfig_bridge0="inet 10.0.0.1/24"

10.0.0.1 acts as the
  • primary nameserver for the host and jails
  • dnsmasq listen address to serve DHCP to the jails
  • gateway for all jails
It needed filtering on the bridge to prevent jail DHCP messages from exiting em0 and getting their IPs assigned by the outer network.

Code: 
# sysctl -a | grep net.link.bridge.pfil
net.link.bridge.pfil_local_phys: 1
net.link.bridge.pfil_member: 1
net.link.bridge.pfil_bridge: 1
net.link.bridge.pfil_onlyip: 0

# /etc/pf.conf
block quick on $ext_if proto { udp, tcp } from $int_if to $int_if port { 67, 68, 69 }
block quick on $int_if proto { udp, tcp } from $ext_if to $ext_if port { 67, 68, 69 }

On boot, the jails get IPs in 192.168.1.0/24 initially and have to be restarted to get 10.0.0.0/24.

If I remove em0 from the bridge to separate the internal network completely, will pf still be able to pass traffic between them? Or is there a better network design for this sort of thing?
 
ccammack said:
If I remove em0 from the bridge to separate the internal network completely, will pf still be able to pass traffic between them?
Click to expand...

What about "classic" config often used on VDS?

example:
ext_if (external ip) --- bridge0 (192.168.x.1/24) --- jails/VMs (192.168.x.0/24 addresses) and nat+rdr by pf?
 
CeXP1917 said:
ext_if (external ip) --- bridge0 (192.168.x.1/24) --- jails/VMs (192.168.x.0/24 addresses) and nat+rdr by pf?
Click to expand...
My current "working" version looks similar, I think, except I'm new to pf haven't used rdr rules yet.
  • the host gets its IP 192.168.1.181 for em0 from the upstream LAN
  • the bridge has em0 as its only member and I have assigned the bridge IP 10.0.0.1
  • dnsmasq runs on the host, listens on 10.0.0.1, and gives IPs in 10.0.0.0/24 to the VNET jails when they are created and join the bridge
  • both host and jails use 10.0.0.1 as their first name server, so both can query dnsmasq DNS to look up jail names
What about "classic" config often used on VDS?
Click to expand...
If VDS is "Virtual Dedicated Server," that sounds close to what I'm trying to do. (I was thinking of it as a self-contained LAN with the host as router for an internal virtual network of jails/VMs). I'll look for sample VDS configurations to see how it's done.

Do you think it's better to keep em0 as a member of the bridge or should I break that connection? I've seen posts here saying that you should either give the bridge members or an IP address, but generally not both. My "working" version does both.

Can pf rdr rules still redirect traffic between ext_if and the bridge even if ext_if is not a member of the bridge?
 
ccammack said:
Do you think it's better to keep ext_if as a member of the bridge or should I break that connection?
Click to expand...
Its better to have bridge and ext_if separated.
ccammack said:
Can pf still redirect traffic between ext_if and the bridge even if ext_if is not a member of the bridge?
Click to expand...
Yes, by simple nat/rdr configuration.

From my VDS:

part of /etc/rc.conf:
... gateway_enable="YES" # internal network cloned_interfaces="bridge0" # just bridge ifconfig_bridge0="inet 172.16.99.1 netmask 255.255.255.0 up" ... # firewall pf_enable="YES" pf_flags="" pf_rules="/etc/pf.conf" pflog_enable="YES" pflog_flags="" ...

part /etc/sysctl.conf:
net.inet.ip.forwarding=1


part of /etc/pf.conf (concept):
ext_if = "vtnet0" # ext if in VM, e.g. WAN for VDS ... set skip on bridge0 # I dont need filtering on internal bridge ... rdr on $ext_if inet proto tcp from any to any port 2222 -> $jail_ip port 22 # redirecting ssh to jail ... nat pass on $ext_if from bridge0:network to any -> ($ext_if) # nat for jails ... pass in on $ext_if proto tcp to $jail_ip port 22 keep state # allow redirected traffic to jail on ext if ...

Ed.: I forget important thing: jails must be VNET jails.
 
You must log in or register to reply here.
Back
Top