Unable to install packages in jail after updating to FreeBSD 14

N

NiftySpace

I have a server running numerous jails which I set up using Bastille. All of my jails use VNETs for networking and are attached to a shared bridge alongside my network interface bge0. Ever since updating to FreeBSD 14, my jails are unable to install packages, even though they do have network connectivity.

Here you can see that the jail times out when trying to bootstrap the package manager:

Code: 
root@nextcloud:~ # pkg-static bootstrap -f
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
pkg(8) is already installed. Forcing reinstallation through pkg(7).
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly, please wait...
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly/Latest/pkg.txz: Operation timed out
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.

It's not a DNS issue, though, because we can resolve the name, however we are unable to talk to the address:

Code: 
root@nextcloud:~ # host pkg.FreeBSD.org
pkg.FreeBSD.org is an alias for pkgmir.geo.FreeBSD.org.
pkgmir.geo.FreeBSD.org has address 192.158.248.8
pkgmir.geo.FreeBSD.org has IPv6 address 2001:500:6b:d::50:2
pkgmir.geo.FreeBSD.org mail is handled by 0 .
root@nextcloud:~ # ping pkg.FreeBSD.org
PING pkgmir.geo.FreeBSD.org (192.158.248.8): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- pkgmir.geo.FreeBSD.org ping statistics ---
9 packets transmitted, 0 packets received, 100.0% packet loss
root@nextcloud:~ # ping 192.158.248.8
PING 192.158.248.8 (192.158.248.8): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- 192.158.248.8 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

What's weird about this is that the jail does have Internet connectivity. For example, we can ping various IP addresses:

Code: 
root@nextcloud:~ # ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=55 time=16.701 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=13.684 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=10.083 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=55 time=13.356 ms
^C
--- 1.1.1.1 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 10.083/13.456/16.701/2.344 ms
root@nextcloud:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=116 time=13.406 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=116 time=9.935 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=116 time=13.833 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=116 time=19.241 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 9.935/14.104/19.241/3.329 ms

To get a few common things out the way:
  • There is no firewall running on the jail
  • There is no firewall running on the host
  • The host is able to ping the jail and vice-versa
  • The host and jail have the same resolv.conf
As you can see here, the host can talk to the FreeBSD package server, but the jail cannot:

Code: 
user@home:~ $ ping pkg.FreeBSD.org
PING pkgmir.geo.FreeBSD.org (192.158.248.8): 56 data bytes
64 bytes from 192.158.248.8: icmp_seq=0 ttl=54 time=13.707 ms
64 bytes from 192.158.248.8: icmp_seq=1 ttl=54 time=14.729 ms
64 bytes from 192.158.248.8: icmp_seq=2 ttl=54 time=12.332 ms
64 bytes from 192.158.248.8: icmp_seq=3 ttl=54 time=24.475 ms
^C
--- pkgmir.geo.FreeBSD.org ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 12.332/16.311/24.475/4.790 ms
user@home:~ $ doas bastille console nextcloud
[budget]:
root@nextcloud:~ # ping pkg.FreeBSD.org
PING pkgmir.geo.FreeBSD.org (192.158.248.8): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- pkgmir.geo.FreeBSD.org ping statistics ---
9 packets transmitted, 0 packets received, 100.0% packet loss
 
Here is the relevant network configuration from ifconfig on the host:

Code: 
bge0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=c0099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
    ether b8:cb:29:d2:4f:55
    inet 192.168.0.100 netmask 0xffffff00 broadcast 192.168.0.255
    inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255
    inet6 fe80::bacb:29ff:fed2:4f55%bge0 prefixlen 64 scopeid 0x1
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
...
bge0bridge: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=0
    ether 58:9c:fc:10:cc:31
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    ... others removed ...
    member: e0a_bastille3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 2000
    member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
...
e0a_bastille3: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: vnet host interface for Bastille jail nextcloud
    options=8<VLAN_MTU>
    ether 02:20:9b:d2:4f:55
    hwaddr 02:eb:e6:c5:fe:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

And from the jail:

Code: 
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 0e:20:9b:d2:4f:55
    hwaddr 02:eb:e6:c5:fe:0b
    inet 192.168.0.104 netmask 0xff000000 broadcast 192.255.255.255
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

(The interface e0b_bastille3 was given to the jail.)
 
I second the netmask observation; I guess it should be 255.255.255.0, and not 255.0.0.0.

I saw a similar issue while going from 13.1-RELEASE to 14.0-RELEASE on my Internet router today.

Internal hosts using the router couldn't access Internet IPs beginning with 192, including GitHub (and the package servers, like you're seeing).

So, on 13.1-RELEASE these rc.conf lines resulted in a 255.255.255.0 netmask (or maybe 255.255.255.255; I forget) on the aliased IP:

Code: 
ifconfig_lan0="inet 192.168.32.48 netmask 255.255.255.0"
ifconfig_lan0_alias1="inet 192.168.32.1"

But on 14.0-RELEASE, the same rc.conf lines resulted in a 255.0.0.0 netmask on the aliased IP, breaking routing to all Internet IPs beginning with 192.

To fix it I set the netmask on the aliased IP explicitly, like this:

Code: 
ifconfig_lan0="inet 192.168.32.48 netmask 255.255.255.0"
ifconfig_lan0_alias1="inet 192.168.32.1 netmask 255.255.255.0"
 
The OP seems to be using an IP I would not.
192.168.0.0
There is special significance to both 0 and 255.
Avoid both.
There is a reason all ClassC stuff starts at 192.168.1.1.

robroy said:
So, on 13.1-RELEASE these rc.conf lines resulted in a 255.255.255.0 netmask (or maybe 255.255.255.255; I forget)
Click to expand...
255.255.255.0 for Class C address range.

You can shorthand it.
robroy said:
ifconfig_lan0="inet 192.168.32.48/24"
ifconfig_lan0_alias1="inet 192.168.32.1/24"
Click to expand...
 
Thank you junkyalleycat and robroy for your help... the netmask seems to have been the issue! I've manually edited the rc.conf for all of my jails and restarted them, and that seems to have restored connectivity. I can update the jails and their packages now. (Well, except for one jail that is still experiencing issues. I'm not sure why that is.)

And thank you Phishfry for the note about the addresses... I'll just bump up the last byte of the IP address of my jail host and jails by 1 to get it into the appropriate range. However, based on the rudimentary IP knowledge I have, it should be possible to use an address ending in .0 if you use the correct netmask, correct? As I understand it, the terminology of class A/B/C networks is obsolete and no longer relevant.
 
NiftySpace said:
it should be possible to use an address ending in .0 if you use the correct netmask, correct?
Click to expand...
As long as not all host bits are zero. So, 192.168.1.0 is fine for a host IF the netmask is 255.255.0.0 for example. All host bits 0 is the network address, all host bits 1 is the broadcast address. Those are two addresses you cannot assign to a host.
 
Hi all,
and thank you for that thread.

  • I currently had the same error after updating the host system to 14.0-RELEASE
  • Bastille is on the latest version 0.10.20231125
  • I do not use the VNET approach but the loopback "bastille0" approach.
  • I have pf disabled (temporarily to avoid that this is the issue)

In my case the cause was different.
I had to update my usr/local/etc/bastille/bastille.conf file. This was an old one and Mr. Edwards updated some stuff meanwhile.
I was missing the entry
## pf configuration path
bastille_pf_conf="/etc/pf.conf" ## default: "/etc/pf.conf"
Click to expand...
... and maybe some more - did not compare precisely.
After copying the latest bastille.sample.conf, to bastille.conf, and configuring it to my needs (zfs mostly), everything now works as expected

Maybe that helps someone running into the same issue...
 
You must log in or register to reply here.
Back
Top