jails Troubleshooting network connectivity inside Bastille nat jail

B

brendans-bits

Hi all,
I am struggling to troubleshoot network connectivity issues on a BastilleBSD nat'd jail created using these instructions: https://docs.bastillebsd.org/en/latest/chapters/networking.html#loopback-bastille0
In the container I can resolve DNS queries (e.g. pinging a url returns a valid external IP address) but cannot ping external URLs (I have allow.raw_sockets; set for this container) and commands such as pkg update hang. I can ping the IP addresses of the two interfaces only, but cannot ping anything else.
(Note: On the host I have two network interfaces (from a time I was using VMs).)

I thought that this might be the problem, but I can't add a default route:
Code: 
root@azkaban:~ # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.0.1           link#4             UH     bastille

root@azkaban:~ # route add default 192.168.101.123
route: writing to routing socket: Operation not permitted

What can I check next to locate the source of the network issues?

Bastille created jail.conf:
Code: 
# cat /usr/local/bastille/jails/azkaban/jail.conf
azkaban {
  devfs_ruleset = 4;
  enforce_statfs = 2;
  exec.clean;
  exec.consolelog = /var/log/bastille/azkaban_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = azkaban;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/azkaban/fstab;
  path = /usr/local/bastille/jails/azkaban/root;
  securelevel = 2;
  allow.raw_sockets; #<- This is the only line I added
  interface = bastille0;
  ip4.addr = 10.0.0.1;
  ip6 = disable;
}

Relevant lines from host rc.conf:
Code: 
hostname="freebsd-server"
ifconfig_re0="DHCP"                                                                                                                                                                                  
ifconfig_igb0="DHCP"                                                                                                                                                                                 
ifconfig_re0_ipv6="inet6 accept_rtadv" 
gateway_enable="YES"
bastille_enable="YES"
bastille_zfs_pool="storage/bastille"
cloned_interfaces="lo1"
ifconfig_lo1_name="bastille0"

Output of ifconfig on the host:
Code: 
# ifconfig
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether xxxxxx
        inet 192.168.101.123 netmask 0xffffff00 broadcast 192.168.101.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether xxxx
        inet6 fe80::aaa1:59ff:fe41:2c2b%re0 prefixlen 64 scopeid 0x2
        inet6 2403:5815:9651:0:aaa1:59ff:fe41:2c2b prefixlen 64 autoconf
        inet 192.168.101.2 netmask 0xffffff00 broadcast 192.168.101.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bastille0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 fe80::1%bastille0 prefixlen 64 scopeid 0x4
        inet 10.0.0.1 netmask 0xffffffff
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Output of ifconfig in the container:
Code: 
root@azkaban:~ # ifconfig
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether xxxx
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether xxxx
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        groups: lo
bastille0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.0.0.1 netmask 0xffffffff
        groups: lo

Host pf.conf:
Code: 
ext_if="igb0"

set block-policy return
scrub in on $ext_if all fragment reassemble
set skip on lo

table <jails> persist
nat on $ext_if from <jails> to any -> ($ext_if:0)
rdr-anchor "rdr/*"

block in all
pass out quick keep state
antispoof for $ext_if inet

pass in inet proto tcp from any to any port ssh flags S/SA keep state
pass in inet proto tcp from any to any port {2001}

I have ssh running in the container:
Code: 
root@azkaban:~ # sockstat -4
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     sshd       4342  4  tcp4   10.0.0.1:22           *:*

And bastille is redirecting ports:
Code: 
# bastille rdr azkaban list
rdr pass on igb0 inet proto tcp from any to any port = 2001 -> 10.0.0.1 port 22

Code: 
# bastille list -a
 JID      State  IP Address  Published Ports  Hostname  Release          Path
 azkaban  Up     10.0.0.1    tcp/2001:22      azkaban   13.2-RELEASE-p4  /usr/local/bastille/jails/azkaban/root

I can ssh into the container from the host, but not from anywhere else on my network:
Code: 
[brendan@bookcase ~]$ nc 192.168.101.123 2001
Ncat: Connection refused.
[brendan@bookcase ~]$ nc 192.168.101.2 2001
Ncat: Connection refused.
 
On the host both igb0 and re0 are on the same 192.168.101.0/24 subnet. That's always going to fail.
 
You must log in or register to reply here.
Back
Top