Bridge doesn't setup correctly on boot

mmccoo · Sep 13, 2016

Problem summary

When booting, a bridge is created, but the two member networks are not added.
if I just do service netif restart, the members are added and the bridge is fine
but wlan0 is now open instead of secured.
if I then do service hostapd restart, the wlan0 is back to protected.

After this series of steps, everything appears to be as I want it. Still, I'd prefer for it to just reboot into this state. Hopefully someone can tell me what I'm doing wrong. The config files are fine enough that the series of restarts yields a desired state but not enough to just work

Thank you in advance for your time and attention.
Miles

Hardware:

I'm running 11.0-ALPHA6 on a Raspberry pi 2
cheap RT5370 wireless adapter.
The ethernet port is connected to my house router through a powerline adapter.

What I'm trying to accomplish

a range extender upstairs (lots of concrete/stone in buildings here in Germany)
a simple backup/file server by connecting a usb drive to it.
I'm using bridge mode because I believe that Plex really wants to be on the same subnet as my firestick. Boot type stuff was fine when using gateway mode, but plex ran slow.

Relevant files

My rc.conf file

Code:

hostname="malcolm"
ntpd_enable="YES" # keep the system regular
ntpd_sync_on_start="YES"
sshd_enable="YES"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
growfs_enable="YES"

wlans_run0="wlan0"
create_args_wlan0="wlanmode hostap"
ifconfig_wlan0="ssid rapi mode 11g channel 3 up"
hostapd_enable="YES"

# allow packets to be passed from one network interface to another
gateway_enable="YES"            # Enable as LAN gateway

cloned_interfaces="bridge0"
ifconfig_bridge0="addm wlan0 addm ue0 up"

ifconfig_ue0="DHCP"

## PF firewall
pf_enable="YES"                 # Enable PF (load module if required)
pf_rules="/etc/pf.conf"         # rules definition file for pf
pf_flags=""                     # additional flags for pfctl start up
pflog_enable="YES"              # start pflogd(8)
pflog_logfile="/var/log/pflog"  # where pflogd should store the logfile
pflog_flags=""                  # additional flags for pflogd start up

hostapd.conf

Code:

interface=wlan0
debug=1
ctrl_interface=/var/run/hostapd
ctrl_interface_group=wheel
ssid=rapi
wpa=2
wpa_passphrase=notactuallymypasscode
wpa_key_mgmt=WPA-PSK
wpa_pairwise=CCMP

initial ifconfig (note bridge members are missing and wlan0 is secured).

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
        ether b8:27:eb:03:50:49
        inet 192.168.0.87 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:1b:5b:e1:94:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 7c:dd:90:69:59:22
        groups: wlan
        ssid rapi channel 3 (2422 MHz 11g) bssid 7c:dd:90:69:59:22
        regdomain FCC country US authmode WPA2/802.11i privacy MIXED
        deftxkey 2 AES-CCM 2:128-bit txpower 30 scanvalid 60 protmode CTS wme
        dtimperiod 1 -dfs
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
        status: running
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33184
        groups: pflog

After restarting netif (bridge is fine, wlan is broken).

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ue0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
        ether b8:27:eb:03:50:49
        inet 192.168.0.87 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wlan0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 7c:dd:90:69:59:22
        groups: wlan
        ssid rapi channel 3 (2422 MHz 11g) bssid 7c:dd:90:69:59:22
        regdomain FCC country US authmode OPEN privacy OFF txpower 30
        scanvalid 60 protmode CTS wme dtimperiod 1 -dfs
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
        status: running
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:1b:5b:e1:94:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: ue0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55
        member: wlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 3 priority 128 path cost 370370
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

After restarting hostapd (bridge and wlan are now fine... I think).

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ue0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
        ether b8:27:eb:03:50:49
        inet 192.168.0.87 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33184
        groups: pflog
wlan0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 7c:dd:90:69:59:22
        groups: wlan
        ssid rapi channel 3 (2422 MHz 11g) bssid 7c:dd:90:69:59:22
        regdomain FCC country US authmode WPA2/802.11i privacy MIXED
        deftxkey 2 AES-CCM 2:128-bit txpower 30 scanvalid 60 protmode CTS wme
        dtimperiod 1 -dfs
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
        status: running
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:1b:5b:e1:94:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: ue0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55
        member: wlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 3 priority 128 path cost 370370
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

SirDice · Sep 13, 2016

After a quick look I can't find anything wrong with your configurations. It might be an issue with the order of things during boot, maybe one part isn't waiting long enough before starting the other bits. Are you able to do a verbose boot? That might shed some light on things.

mmccoo · Sep 14, 2016

I added this to loader.conf.local:

Code:

verbose_loading="YES"
boot_verbose="YES"

through it didn't seem to add much.

In any case, this is what seems to be the relevant part of /var/log/messages

(when removing this file, how do I convince syslog et al to create a new one? other than reboot)

Code:

Sep 14 19:45:23 malcolm kernel: random: harvesting attach, 8 bytes (4 bits) from miibus0
Sep 14 19:45:23 malcolm kernel: ue0: <USB Ethernet> on smsc0
Sep 14 19:45:23 malcolm kernel: ue0: bpf attached
Sep 14 19:45:23 malcolm kernel: ue0: Ethernet address: b8:27:eb:03:50:49
Sep 14 19:45:23 malcolm kernel: ugen0.4: <Ralink> at usbus0
Sep 14 19:45:23 malcolm kernel: random: unblocking device.
Sep 14 19:45:23 malcolm kernel: bridge0: bpf attached
Sep 14 19:45:23 malcolm kernel: bridge0: Ethernet address: 02:1b:5b:e1:94:00
Sep 14 19:45:23 malcolm kernel: smsc0: chip 0xec00, rev. 0002
Sep 14 19:45:23 malcolm kernel: ue0: link state changed to DOWN
Sep 14 19:45:23 malcolm kernel: ue0: link state changed to UP
Sep 14 19:45:23 malcolm kernel: wlan: <802.11 Link Layer>
Sep 14 19:45:23 malcolm kernel: run0: <1.0> on usbus0
Sep 14 19:45:23 malcolm kernel: run0: MAC/BBP RT5390 (rev 0x0502), RF RT5370 (MIMO 1T1R), address 7c:dd:90:69:59:22
Sep 14 19:45:23 malcolm kernel: run0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
Sep 14 19:45:23 malcolm kernel: run0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
Sep 14 19:45:23 malcolm kernel: random: harvesting attach, 8 bytes (4 bits) from run0
Sep 14 19:45:23 malcolm kernel: wlan0: bpf attached
Sep 14 19:45:23 malcolm kernel: ieee80211_load_module: load the wlan_amrr module by hand for now.
Sep 14 19:45:23 malcolm kernel: wlan0: bpf attached
Sep 14 19:45:23 malcolm kernel: wlan0: Ethernet address: 7c:dd:90:69:59:22
Sep 14 19:45:23 malcolm kernel: firmware: 'runfw' version 1: 8192 bytes loaded at 0xc41f6628
Sep 14 19:45:23 malcolm kernel: run0: firmware RT3071 ver. 0.33 loaded
Sep 14 19:45:23 malcolm kernel: pflog0: bpf attached
Sep 14 19:45:23 malcolm kernel: pflog0: promiscuous mode enabled
Sep 14 19:45:25 malcolm ntpd[712]: ntpd 4.2.8p8-a (1): Starting
Sep 14 19:45:25 malcolm ntpd[713]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): good hash signature
Sep 14 19:45:25 malcolm ntpd[713]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): loaded, expire=2016-12-01T00:00:00Z last=2015-07-01T00:00:00Z ofs=36
Sep 14 19:45:57 malcolm kernel: .
Sep 14 19:45:59 malcolm kernel: firmware: 'runfw' version 1: 8192 bytes loaded at 0xc41f6628
Sep 14 19:46:00 malcolm kernel: run0: firmware RT3071 ver. 0.33 loaded

Looks like the bridge is created before wlan0. Seems like bridge should always be last, no? I suppose the next step is adding logger calls in /etc/rc.d/netif and /etc/network.subr to figure out why the order is "wrong".

Alternately, is there a different config file where I could create the wlan0 interface? once everything is booted, the cmdline ifconfig commands do work.

I'd really like things to be correct on reboot without intervention. Hopefully that will be a rare occurrence, but I'm just picturing trying to walk my wife through it when I'm not home.

Miles

SirDice · Sep 15, 2016

mmccoo said:
looks like the bridge is created before wlan0.

Ah, that explains why it's not correct just after boot.

Seems like bridge should always be last, no?

That would make more sense, yes.

It's easy to check the order with rcorder(8):

Code:

root@molly:~ # rcorder /etc/rc.d/*
/etc/rc.d/growfs
/etc/rc.d/sysctl
/etc/rc.d/hostid
/etc/rc.d/zvol
/etc/rc.d/dumpon
/etc/rc.d/ddb
/etc/rc.d/geli
/etc/rc.d/gbde
/etc/rc.d/ccd
/etc/rc.d/swap
/etc/rc.d/fsck
/etc/rc.d/root
/etc/rc.d/mdconfig
/etc/rc.d/hostid_save
/etc/rc.d/mountcritlocal
/etc/rc.d/zfs
/etc/rc.d/var
/etc/rc.d/cleanvar
/etc/rc.d/FILESYSTEMS
/etc/rc.d/ldconfig
/etc/rc.d/kldxref
/etc/rc.d/kld
/etc/rc.d/addswap
/etc/rc.d/adjkerntz
/etc/rc.d/atm1
/etc/rc.d/hostname
/etc/rc.d/ip6addrctl
/etc/rc.d/netoptions
/etc/rc.d/random
/etc/rc.d/sppp
/etc/rc.d/ipfilter
/etc/rc.d/ipnat
/etc/rc.d/ipfs
/etc/rc.d/serial
/etc/rc.d/iovctl
/etc/rc.d/netif
/etc/rc.d/devd
/etc/rc.d/zfsd
/etc/rc.d/ipsec
/etc/rc.d/atm2
/etc/rc.d/pfsync
/etc/rc.d/pflog
/etc/rc.d/pf
/etc/rc.d/stf
/etc/rc.d/ppp
/etc/rc.d/routing
/etc/rc.d/ipfw
/etc/rc.d/netwait
/etc/rc.d/resolv
/etc/rc.d/local_unbound
/etc/rc.d/nsswitch
/etc/rc.d/routed
/etc/rc.d/rtsold
/etc/rc.d/static_ndp
/etc/rc.d/static_arp
/etc/rc.d/bridge
/etc/rc.d/route6d
/etc/rc.d/defaultroute
/etc/rc.d/NETWORKING
/etc/rc.d/mountcritremote
/etc/rc.d/newsyslog
/etc/rc.d/syslogd
/etc/rc.d/ntpdate
/etc/rc.d/rpcbind
/etc/rc.d/devfs
/etc/rc.d/ipmon
/etc/rc.d/kdc
/etc/rc.d/mdconfig2
/etc/rc.d/watchdogd
/etc/rc.d/savecore
/etc/rc.d/archdep
/etc/rc.d/abi
/etc/rc.d/SERVERS
/etc/rc.d/nisdomain
/etc/rc.d/ypserv
/etc/rc.d/ypxfrd
/etc/rc.d/ypupdated
/etc/rc.d/ypbind
/etc/rc.d/ypset
/etc/rc.d/yppasswdd
/etc/rc.d/ypldap
/etc/rc.d/wpa_supplicant
/etc/rc.d/virecover
/etc/rc.d/accounting
/etc/rc.d/nfsclient
/etc/rc.d/amd
/etc/rc.d/atm3
/etc/rc.d/auditd
/etc/rc.d/auditdistd
/etc/rc.d/tmp
/etc/rc.d/cleartmp
/etc/rc.d/ctld
/etc/rc.d/dmesg
/etc/rc.d/hastd
/etc/rc.d/iscsid
/etc/rc.d/iscsictl
/etc/rc.d/keyserv
/etc/rc.d/nfsuserd
/etc/rc.d/gssd
/etc/rc.d/quota
/etc/rc.d/mountd
/etc/rc.d/nfsd
/etc/rc.d/statd
/etc/rc.d/lockd
/etc/rc.d/pppoed
/etc/rc.d/pwcheck
/etc/rc.d/DAEMON
/etc/rc.d/utx
/etc/rc.d/ugidfw
/etc/rc.d/ubthidhci
/etc/rc.d/timed
/etc/rc.d/apm
/etc/rc.d/apmd
/etc/rc.d/bootparams
/etc/rc.d/hcsecd
/etc/rc.d/bthidd
/etc/rc.d/local
/etc/rc.d/lpd
/etc/rc.d/motd
/etc/rc.d/mountlate
/etc/rc.d/nscd
/etc/rc.d/ntpd
/etc/rc.d/powerd
/etc/rc.d/rarpd
/etc/rc.d/rctl
/etc/rc.d/sdpd
/etc/rc.d/rfcomm_pppd_server
/etc/rc.d/rtadvd
/etc/rc.d/rwho
/etc/rc.d/LOGIN
/etc/rc.d/syscons
/etc/rc.d/swaplate
/etc/rc.d/sshd
/etc/rc.d/sendmail
/etc/rc.d/cron
/etc/rc.d/jail
/etc/rc.d/localpkg
/etc/rc.d/securelevel
/etc/rc.d/power_profile
/etc/rc.d/othermta
/etc/rc.d/nfscbd
/etc/rc.d/natd
/etc/rc.d/msgs
/etc/rc.d/moused
/etc/rc.d/mixer
/etc/rc.d/kpasswdd
/etc/rc.d/kfd
/etc/rc.d/kadmind
/etc/rc.d/ipropd_slave
/etc/rc.d/ipropd_master
/etc/rc.d/inetd
/etc/rc.d/hostapd
/etc/rc.d/gptboot
/etc/rc.d/geli2
/etc/rc.d/ftpd
/etc/rc.d/ftp-proxy
/etc/rc.d/dhclient
/etc/rc.d/bsnmpd
/etc/rc.d/bluetooth
/etc/rc.d/blacklistd
/etc/rc.d/bgfsck
/etc/rc.d/autounmountd
/etc/rc.d/automountd
/etc/rc.d/automount

As can be seen wpa_supplicant is run long after bridge. Even long after NETWORKING. It should probably be run somewhere between netif and bridge.

Try editing /etc/rc.d/wpa_supplicant, and at the top, add a BEFORE line:

Code:

# PROVIDE: wpa_supplicant
# REQUIRE: mountcritremote
# BEFORE: bridge
# KEYWORD: nojail nostart

The REQUIRE line might pose a problem though. I'm not really sure why wpa_supplicant is depending on mountcritremote. I would have expected REQUIRE: netif.

mmccoo · Sep 15, 2016

That makes sense. I tried two experiments, both of which didn't work.

First the one you suggested, wpa_supplicant to depend on netif and be before bridge. Still too late.
I also tried adding hostapd. No better luck. The run0 stuff is basically the last thing listed in /var/log/messages

How do I know which rc.d file is currently in effect at each point in the messages file or on the terminal?

I do have something that does work, though it feels like a hack.

I created the file /etc/rc.d/rapi_bridge:

Code:

#!/bin/sh

# PROVIDE: rapi_bridge
# REQUIRE: hostapd

. /etc/rc.subr

name="rapi_bridge"
rcvar=`set_rcvar`
start_cmd="rapi_bridge_start"
stop_cmd=":"

load_rc_config $name

rapi_bridge_start()
{
    if checkyesno ${rcvar}; then
      echo adding ue0 and wlan0 to bridge
      ifconfig bridge0 addm ue0 addm wlan0 up
    fi
}

and enabled it in rc.conf

Code:

rapi_bridge_enable=YES

The hack nature of this bugs me, but it does seem to work. If I had a way to know where in the rc flow, the run0 stuff is initialized, I would move bridge after that.

Thank you, SirDice, your thoughts led me down some roads/experiments that were educational. So I'm better off than where I started.

Miles

dj015 · Sep 29, 2018

Sorry to bring back an old thread, but I just had a similar problem with a bridge not adding a USB WiFi dongle on boot, as my wlan0 device was getting created too late.

The way I solved it is by using (what seems to be) the second-phase bridge in /etc/rc.d/bridge, that runs later in the boot sequence. Instead of using "addm" directly in the "ifconfig_bridge0" variable, use the "autobridge_*" variables.

Instead of this:

Bash:

cloned_interfaces="bridge0"
ifconfig_bridge0="inet 192.168.0.1 netmask 255.255.255.0 up addm re0 addm wlan0"

I used this:

Bash:

cloned_interfaces="bridge0"
ifconfig_bridge0="inet 192.168.0.1 netmask 255.255.255.0 up"
autobridge_interfaces="bridge0"
autobridge_bridge0="re0 wlan0"

ethoms · Dec 2, 2019

I have a similar issue with setting up a WiFi Access Point which bridges a wired interface with a wireless interface. The bridge doesn't come up after boot. I solved it by adding ifconfig bridge create to /etc/rc.local. That way the bridge gets created last, once the wired interface acquires it's DHCP address and the wireless interface is setup by hostapd.

@OP: Anything that fixes stuff not working after boot can go into /etc/rc.local. It sucks that we have to do it, but nothings perfect, even in FreeBSD.

Here's my /etc/rc.conf:

Code:

...
# NETWORK
hostname="wifi-ap1.example.com"
gateway_enable="YES"
cloned_interface="bridge0"
ifconfig_re0="DHCP"
ifconfig_bridge0="addm re0 addm wlan0 up"
wlans_ath0="wlan0"
create_args_wlan0="wlanmode hostap country ZA txpower 32"
ifconfig_wlan0="mode 11na channel 165:ht/20"    # 5GHz recommended
...
hostapd_enable="YES"
hostapd_program="/usr/local/sbin/hostapd"
...

and my /etc/rc.local:

Code:

ifconfig bridge create

rococo69 · Jul 4, 2023

I had the same issue with a rpi3 B bridging the onboard ethernet and a external USB ethernet adapter. I ended up doing the rc.local aproach.

/etc/rc.conf:

Code:

ifconfig_ue0="inet 192.168.1.45 netmask 255.255.255.0"
defaultrouter="192.168.1.1"
cloned_interfaces="bridge0"

/etc/rc.local:

Code:

ifconfig bridge0 addm ue0 up
ifconfig bridge0 addm ue1 up
ifconfig ue0 up
ifconfig ue1 up

Bridge doesn't setup correctly on boot

mmccoo

SirDice

Administrator

mmccoo

SirDice

Administrator

mmccoo

dj015

ethoms

rococo69