QUESTION
As I understand it, we are supposed to start treating bridges as if they were real hardware switches. So we shouldn't be attaching addresses to the bridge. However, in my simple tests, when I remove the address, the jails that I've attached to the bridge can no longer "see" the bridge, so ping and other traffic cannot pass from the jail to the bridge and then out the host interface. With the same setup, if I re-create everything (jails/bridge) but with the bridge using the 10.0.0.1 jail gateway address, then the jail can "see" the bridge and ping and other traffic can pass to the bridge and out the host interface.
* https://blog.feld.me/posts/2026/02/using-new-bridges-freebsd-15/
* https://www.siberoloji.com/how-to-bridge-two-network-interfaces-on-freebsd/
* https://docs.freebsd.org/en/books/handbook/network/ (strangely, I don't see a section on bridges here...)
So what's the right way to do this now? Note that I eventually plan to have multiple network segments sending traffic over the host interface. I'm guessing that I need/want a separate bridge for jails in separate network segments: 10.0.1.0/24, 10.0.2.0/24, etc.
EXAMPLE: error case
host: 192.168.0.2/24
host interface: em0
jail: 10.0.0.2/25
jail interface: 10_0_0_2b
bridge0:
members: em0, 10_0_0_2a
DETAILS
Host:
The bridge has both em0 and the jail epair. Looks correct to me.
Jail:ifconfig
The jail has an epair and seems to be configured correctly for a /25 network.
jail:netstat
I think 10.0.0.1 is the correct gateway for this subnet.
jail:arp
But this the core problem, ihmo. The jail's vnet cannot arp the 10.0.0.1 address. If I created the bridge with this address on it, then all of this works fine. So what am I supposed to do? Should I add 10.0.0.1 as an alias to the bridge or to em0? or do something else?
As I understand it, we are supposed to start treating bridges as if they were real hardware switches. So we shouldn't be attaching addresses to the bridge. However, in my simple tests, when I remove the address, the jails that I've attached to the bridge can no longer "see" the bridge, so ping and other traffic cannot pass from the jail to the bridge and then out the host interface. With the same setup, if I re-create everything (jails/bridge) but with the bridge using the 10.0.0.1 jail gateway address, then the jail can "see" the bridge and ping and other traffic can pass to the bridge and out the host interface.
* https://blog.feld.me/posts/2026/02/using-new-bridges-freebsd-15/
* https://www.siberoloji.com/how-to-bridge-two-network-interfaces-on-freebsd/
* https://docs.freebsd.org/en/books/handbook/network/ (strangely, I don't see a section on bridges here...)
So what's the right way to do this now? Note that I eventually plan to have multiple network segments sending traffic over the host interface. I'm guessing that I need/want a separate bridge for jails in separate network segments: 10.0.1.0/24, 10.0.2.0/24, etc.
EXAMPLE: error case
host: 192.168.0.2/24
host interface: em0
jail: 10.0.0.2/25
jail interface: 10_0_0_2b
bridge0:
members: em0, 10_0_0_2a
DETAILS
Host:
The bridge has both em0 and the jail epair. Looks correct to me.
Bash:
em0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=8500bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS>
ether 08:00:27:7f:a5:81
inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255
inet6 fe80::a00:27ff:fe7f:a581%em0 prefixlen 64 scopeid 0x1
inet6 2601:602:8100:d9a0:a00:27ff:fe7f:a581 prefixlen 64 autoconf pltime 306212 vltime 306212
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33152
options=0
groups: pflog
bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: jails-bridge
options=10<VLAN_HWTAGGING>
ether 58:9c:fc:10:7e:44
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
bridge flags=0<>
member: 10.0.0.2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 5 priority 128 path cost 2000 vlan protocol 802.1q
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 1 priority 128 path cost 20000 vlan protocol 802.1q
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
10.0.0.2a: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: vnet-loadbalancer_lb_10_0_0_2
options=20000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6>
ether 58:9c:fc:10:78:27
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>Jail:ifconfig
The jail has an epair and seems to be configured correctly for a /25 network.
Bash:
toddg@server_a ~ $ sudo jexec loadbalancer_lb_10_0_0_2 ifconfig
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0 metric 0 mtu 33152
options=0
groups: pflog
10.0.0.2b: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=20000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6>
ether 58:9c:fc:10:78:1c
inet 10.0.0.2 netmask 0xffffff80 broadcast 10.0.0.127
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>jail:netstat
I think 10.0.0.1 is the correct gateway for this subnet.
Bash:
toddg@server_a ~ $ sudo jexec loadbalancer_lb_10_0_0_2 netstat -rn
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 10.0.0.1 UGS 10.0.0.2b
10.0.0.0/25 link#6 U 10.0.0.2b
10.0.0.2 link#7 UHS lo0
127.0.0.1 link#7 UH lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 link#7 URS lo0
::1 link#7 UHS lo0
::ffff:0.0.0.0/96 link#7 URS lo0
fe80::%lo0/10 link#7 URS lo0
fe80::%lo0/64 link#7 U lo0
fe80::1%lo0 link#7 UHS lo0
ff02::/16 link#7 URS lo0jail:arp
But this the core problem, ihmo. The jail's vnet cannot arp the 10.0.0.1 address. If I created the bridge with this address on it, then all of this works fine. So what am I supposed to do? Should I add 10.0.0.1 as an alias to the bridge or to em0? or do something else?
Bash:
toddg@server_a ~ $ sudo jexec loadbalancer_lb_10_0_0_2 arp -na
? (10.0.0.2) at 58:9c:fc:10:78:1c on 10.0.0.2b permanent [ethernet]
[/code
[B]SYSTEM[/B]
15.0-RELEASE