Dual stack (ipv4 & ipv6) NAT Jails (non-VNET)

C

Corwynt

Hi,

I've got FreeBSD 15.1-RELEASE on a host and running Bastille with the non-VNET type jails (using bastille0 cloned from lo1). I'm using a RFC1918 range different from my internal LAN for the IPv4 side of the jails, and for IPv6, I generated a Unique Link Local address (ULA) for the jail. The jail runs unbound for DNS resolution so I wanted it to respond on both IPv4 and IPv6 because my ISP is dual stack as well.

When I start up the jail with bastille, it complains with:
Code: 
[unbound]:
route: route has not been found
unbound: created
IPv4 tcp/53:5353 on igc0,wg0
IPv6 tcp/53:5353 on igc0,wg0
IPv4 udp/53:5353 on igc0,wg0
IPv6 udp/53:5353 on igc0,wg0

However, it still comes up and it responds. (I have unbound listening on port 5353 and bastille port redirection of 53 on the host to go to the jail on port 5353 as seen above. It's listening on igc0 which is the host's ethernet port and wg0, a wireguard interface.)

All of it's working OK for me, but I'm puzzled over the "route: route has not been found" message.

Here's the jail.conf:

Code: 
unbound {
  enforce_statfs = 2;
  devfs_ruleset = 4;
  exec.clean;
  exec.consolelog = /var/log/bastille/unbound_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = unbound;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/unbound/fstab;
  path = /usr/local/bastille/jails/unbound/root;
  securelevel = 2;
  osrelease = 15.1-RELEASE-p2;

  ip4.addr = bastille0|10.99.0.2;
  ip6.addr = bastille0|fd8f:5898:8b34:0::1/128;
}

In /usr/local/etc/bastille/bastille.conf I don't have bastille_network_gateway or
bastille_network_gateway6 set. (It's just set to "")

Is there a way to get rid of this "route: route has not been found" message? Since I'm just using a single ULA for the jail, there really isn't a gateway per se. Would I set a ipv6 gateway of the host or my router?

Thanks!
 
Hi, do you mind sharing the out put of `bastille -x start MYJAIL`?

Not that in earlier versions you might have to do `bastille start -x MYJAIL`
 
Thanks for the reply! I'll crop the top part of the start since it's just searching through the jails but here's the networking part of it:

Code: 
nd]:                                                                                                                                          
+ bastille config unbound get vnet
+ [ 'not set' '!=' enabled ]
+ bastille config unbound get ip4.addr
+ sed 's/,/ /g'
+ ip4_interfaces='bastille0|10.99.0.2'
+ bastille config unbound get ip6.addr
+ sed 's/,/ /g'
+ ip6_interfaces='bastille0|fd8f:5898:8b34:0::1/128'
+ [ 'bastille0|10.99.0.2' '!=' 'not set' ]
+ echo 'bastille0|10.99.0.2'
+ grep -q '|'
+ echo 'bastille0|10.99.0.2'
+ awk '-F|' '{print $1}'
+ if=bastille0
+ echo 'bastille0|10.99.0.2'
+ awk '-F|' '{print $2}'
+ sed -E 's#/[0-9]+$##g'
+ ip=10.99.0.2
+ ifconfig
+ grep ^bastille0:
+ ifconfig
+ grep -qwF 10.99.0.2
+ route -n get 10.99.0.2
+ grep gateway
+ pfctl -q -t jails -T add 10.99.0.2
+ [ 'bastille0|fd8f:5898:8b34:0::1/128' '!=' 'not set' ]
+ echo 'bastille0|fd8f:5898:8b34:0::1/128'
+ grep -q '|'
+ echo 'bastille0|fd8f:5898:8b34:0::1/128'
+ awk '-F|' '{print $1}'
+ if=bastille0
+ echo 'bastille0|fd8f:5898:8b34:0::1/128'
+ awk '-F|' '{print $2}'
+ sed -E 's#/[0-9]+$##g'
+ ip=fd8f:5898:8b34:0::1
+ ifconfig
+ grep ^bastille0:
+ ifconfig
+ grep -qwF fd8f:5898:8b34:0::1
+ route -6 -n get fd8f:5898:8b34:0::1
+ grep gateway
route: route has not been found
+ [ -s /usr/local/bastille/jails/unbound/zfs.conf ]
+ jail -f /usr/local/bastille/jails/unbound/jail.conf -c unbound
unbound: created
+ [ -s /usr/local/bastille/jails/unbound/rctl.conf ]
+ [ -s /usr/local/bastille/jails/unbound/cpuset.conf ]
+ [ -s /usr/local/bastille/jails/unbound/rdr.conf ]
+ read rules
+ bastille rdr unbound dual igc0,wg0 any any tcp 53 5353
IPv4 tcp/53:5353 on igc0,wg0
IPv6 tcp/53:5353 on igc0,wg0
+ read rules
+ bastille rdr unbound dual igc0,wg0 any any udp 53 5353
IPv4 udp/53:5353 on igc0,wg0
IPv6 udp/53:5353 on igc0,wg0
+ read rules
+ sleep 0

It looks like it's running route -6 -n get fd8f:5898:8b34:0::1 and grep'ing for gateway and not finding it.
Now I checked the host and I can't seem to ping6 outbound to anything. I just get:

ping6: UDP connect: No route to host

My /etc/rc.conf has:

Code: 
ifconfig_igc0="DHCP"
ifconfig_igc0_ipv6="inet6 accept_rtadv"
rtsold_enable="YES"
rtsold_flags="-aF"

Somehow I don't think it's getting the router advertisements? Other machines in the network can connect to the FreeBSD host on ipv6.. but it looks like it's not
getting the gateway from the router.

When I run "ndp -r" I get nothing back.
 
OK maybe this isn't a bastille/jails problem but a SLAAC problem.
For example, on a Linux machine on the same network I can do 'ip neigh' and see my router in there:

2axx:xxxx:xxxx:xxxx::1 dev wlan0 lladdr ... router STALE
fe80::.... dev wlan0 lladdr ... router REACHABLE
and some STALE entries.. so the Linux box is getting router advertisements.

If I disable pf on the FreeBSD host, it still doesn't get router advertisements.
 
I figured it out.I had set gateway_enable="YES" and ipv6_gateway_enable="YES" in my /etc/rc.conf thinking I'd need it for the services in the jails, but that's not really needed.
I only need to set net.inet.ip.forwarding=1 for the wireguard tunnels. So I took how the gateway_enable and ipv6_gateway_enable in /etc/rc.conf and set
net.inet.ip.forwarding=1 in /etc/sysctl.conf and that makes wireguard and the pf NAT happy AND the FreeBSD server accepts the router advertisements now.

ndp -r shows the link local address of my router as it should, and I can now ping ipv6 addresses.

When I bastille start unbound, I no longer get the "route: route has not been found" message.

I suppose, if I was to run wireguard with ipv6 on this host which isn't actually a router, would I have to configure the gateway by hand in /etc/rc.conf?
 
You must log in or register to reply here.
Back
Top