Solved Boot sequencing for USB NICs on servers

gpw928

gpw928

My ZFS server uses a USB Ethernet adapter as it's principal NIC. It enumerates as ue0.

There's a sequencing issue with booting. All the PCIe bus NICs enumerate as expected, but ue0 is unresponsive early in the boot sequence. I have experienced "time to enumerate" issues with USB NICs on Linux systems as well. So I'm not surprised.

The default route is set in the usual way:
Code: 
$ grep ue0 /etc/rc.conf
ifconfig_ue0="inet 192.168.1.27 netmask 255.255.255.0"  # USB3

$ grep default /etc/rc.conf
defaultrouter="192.168.1.254"
However, the default route is simply not set (as it would be if I used a PCIe NIC), and I have to fix that explicitly in /etc/rc.local:
Code: 
unset defaultrouter
[ -r /etc/rc.conf ] && . /etc/rc.conf
[ "X$defaultrouter" = "X" ] && exit 0
netstat -rn | grep -q "^default" && exit 0
route add -net 0.0.0.0/0 $defaultrouter
This causes ongoing problems, as things that need the Internet, just after boot, are not working properly:
Code: 
[sherman.129] $ ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 0.pool.ntp.org. .POOL.          16 p    -   64    0    0.000   +0.000   0.000
 1.pool.ntp.org. .POOL.          16 p    -   64    0    0.000   +0.000   0.000
 127.127.1.0     .LOCL.          10 l   53   64    1    0.000   +0.000   0.000
Ntpd only works is because I have configured a fallback to the hardware clock at stratum 10. The absence of external time servers happens because the default route gets corrected (out of /etc/rc.local) after ntpd starts. It recovers, eventually, but not in time to initially set the clock from external time servers, as I would wish.

There are also problems with the NFS daemon servicing clients, which also recovers, eventually.

I'm looking for an elegant fix. Requiring each NIC on a USB interface to settle before bringing up the interface seems to me to be the right way to go.

I could create an rc script (as a dependency for raising network devices) to do this. I could easily adapt this code I wrote for a Linux system to make sure that its only USB NIC is present:
Code: 
# You need the VID:PID if INTERNET_NIC is on a USB adapter
VIDPIDa=0b95:772a       # 772a ASIX AX88772A Fast Ethernet
VIDPIDb=0b95:1790       # 1790 ASIX AX88179 Gigabit Ethernet
VIDPIDc=0bda:8153       # 8153 Realtek RTL8153 Gigabit Ethernet
VIDPIDd=0bda:8150       # 8150 Realtek RTL8150 Fast Ethernet Adapter
VIDPIDe=0bda:8152       # 8152 Realtek RTL8152 Fast Ethernet Adapter
USBVPLIST="${VIDPIDa}|${VIDPIDb}|${VIDPIDc}|${VIDPIDd}|${VIDPIDe}"
# Wait for USB devices to asynchronously initialise and "appear".
# USBVPLIST contains strings in egrep format, of USB "VID:PID".
# If your Internet connection is through a USB device, you need this.
if [ -n "$USBVPLIST" ]
then
    for i in a b c d e f g h i j k l m n o p q r s t u v w x y z
    do
        lsusb | egrep -q "$USBVPLIST" && break
        sleep 1
    done
fi
Is there a more elegant way on FreeBSD?
 
What about the netwait rc.d directive for ue0?

Netwait on FreeBSD – The Blog

blog.danielisz.org blog.danielisz.org
[SNIP] from /etc/defaults/rc.conf
Code: 
### Network link/usability verification options
netwait_enable="NO"             # Enable rc.d/netwait (or NO)
#netwait_ip=""                  # Wait for ping response from any IP in this list.
netwait_timeout="60"            # Total number of seconds to perform pings.
#netwait_if=""                  # Wait for active link on each intf in this list.
netwait_if_timeout="30"         # Total number of seconds to monitor link state.


The other way with more control is devd and create a /etc/devd/yourdevice.conf
 
The comments in /etc/rc.d/netwait indicate that it's exactly what I was hoping to find:
Code: 
# The netwait script helps handle two situations:
#  - Systems with USB or other late-attaching network hardware which
#    is initialized by devd events.  The script waits for all the
#    interfaces named in the netwait_if list to appear.
#  - Systems with statically-configured IP addresses in rc.conf(5).
#    The IP addresses in the netwait_ip list are pinged.  The script
#    waits for any single IP in the list to respond to the ping.  If your
#    system uses DHCP, you should probably use synchronous_dhclient="YES"
#    in your /etc/rc.conf instead of netwait_ip.
# Either or both of the wait lists can be used (at least one must be
# non-empty if netwait is enabled).
So I added the following to /etc/rc.conf
Code: 
$ grep netwait /etc/rc.conf
netwait_enable="YES"
netwait_timeout="20"
netwait_if="ue0"
And the network settings relating to ue0 now get initialised properly on boot.

There's one niggle that I need to chase down. There's still a fleeting message on the console during boot indicating that there was a problem setting the default route. It happens at a time when the video output switches resolution and it's extremely fleeting -- and does not appear to show up in any of the logs. But the default route is set correctly and everything is working by the time I am able to login. I need to hunt that down.
 
That certainly makes sense. routed comes well before netwait in the rcorder. But I feel the need to investigate the other dependencies, lest I break something else.
 
I added the following to the end of /etc/rc.conf
Code: 
# USB NICs takes time to settle at boot
netwait_enable="YES"
netwait_timeout="10"
netwait_ip="${defaultrouter}"
netwait_if="ue0"
Setting both the interface to wait for and the IP to ping on that interface seems to have resolved all functional issues.

/etc/rc.d/routing runs before /etc/rc.d/netwait, and the routing script still complains when setting the default route, but it's benign -- the default route is planted and works.
 
You must log in or register to reply here.
Back
Top