Solved Networking broken after upgrade to 14.0-RELEASE-p5

JT42

JT42

After upgrading from 13.3 to 14.0, all networking has stopped working. I can ping the local IP address but can't ping anything else via IPv4 or IPv6. The output of ifconfig looks okay to me for interface re0, but maybe I'm missing something.

IMG_2612.JPG


Any thoughts?
 
I had ntpdate_enable="YES" in /etc/rc.conf, and with networking not working, it would hang for several minutes since it couldn't reach the ntp server. So I removed that (hopefully temporarily) to speed up the boot time.

So after I log in, if I do service netif restart I can ping things on my local network but nothing else. And after that, if I do service routing restart, I can thing ping the rest of the world. Doing service -e showed that netif had been started, even before I issued the restart command. But when I reboot, networking stops working again.
 
Thanks. It's FVWM2 made to looks like classic CDE / Motif.

/etc/rc.conf:
Code: 
# Set the host name
hostname="haleru.tonneson.org"

# Configure ipv4
ifconfig_re0="inet 192.168.1.21 netmask 0xffffff00"
defaultrouter="192.168.1.1"

# Configure ipv6
ifconfig_re0_ipv6="inet6 accept_rtadv"

# Enable secure shell
sshd_enable="YES"

# Sync the clock
#ntpdate_enable="YES"

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"

# Start the web server
apache24_enable="YES"
apache24_http_accept_enable="YES"

# Enable ZFS
zfs_enable="YES"

# Disable sendmail
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Enable ssmtp for outgoing mail
ssmtp_enable="YES"

# Enable the AMD built-in GPU and Linux compatibility
kld_list="acpi_video amdgpu linux"

# Enable interprocess communication
dbus_enable="YES"

# Enable the Common Unix Print System
cupsd_enable="YES"

# Invoke the local devfs rule (see /etc/devfs.rules) needed by CUPS.
devfs_system_ruleset="system"

# Enable linux compatibility
linux_enable="YES"
ubuntu_enable="NO"

# Use XDM to start FVWM
xdm_enable="YES"

# Create the RAM disk
ramdisk_enable="YES"
 
Oh, you have a statically configured interface. Is that a USB interface by a chance?

Can you post the screen when booting finishes and it asks for a login (in text mode)?
 
re0 is the built-in ethernet port on the motherboard, not via USB.

I'll disable XDM, restart, and capture the screen.
 
Output of 'netstat -nr' when it's not working:
Code: 
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS         re0
127.0.0.1          link#2             UH          lo0
192.168.1.0/24     link#1             U           re0
192.168.1.21       link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             link#2                        URS         lo0
default                           fe80::fa8f:caff:fe1b:2bec%re0 UG          re0
::1                               link#2                        UHS         lo0
::ffff:0.0.0.0/96                 link#2                        URS         lo0
2605:a601:afeb:d901::/64          link#1                        U           re0
2605:a601:afeb:d901:5a11:22ff:feb8:ea54 link#2                  UHS         lo0
fdaa:8852:88:1::/64               link#1                        U           re0
fdaa:8852:88:1:5a11:22ff:feb8:ea54 link#2                       UHS         lo0
fe80::%lo0/10                     link#2                        URS         lo0
fe80::%re0/64                     link#1                        U           re0
fe80::5a11:22ff:feb8:ea54%lo0     link#2                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         link#2                        URS         lo0

I also ran that when it was working and received the same results.

Another clue: If I remove ntpdate_enable="YES" from /etc/rc.conf, then it appears to work -- I rebooted twice with it removed and networking worked both times without me having to restart any services. When I put it back in and rebooted, networking was borked until I restarted the netif and routing services.

Edit to add: With ntpdate_enable removed, sometimes networking will come up and work just fine, other times it won't.
 
Your window decorations are very familiar. I ran fvwm on FreeBSD, for many years.

/etc/rc.d/ntpdate extracts the names of time servers from /etc/ntp.conf.

It looks like your network is up, but your name server is not translating those time server names to IP addresses.

What happens if you add ntpdate_hosts=216.239.35.4 to /etc/rc.conf?

Can you ping 8.8.8.8? Does nslookup google.com work? What's in /etc/resolv.conf?
 
It look like ipv6 issue.

When you have network problem to diagnose follow those simple steps.

1. Check if the interface have the correct IP address and netmask and it's status is active
2. Check if you have ping to the default gateway IP address (in your case 192.168.1.1 and fe80::fa8f:caff:fe1b:2bec)
3. Check if the router is working, try to ping some host behind the router like 8.8.8.8 and 2001:4860:4860::8888
4. Check if your DNS is reachable, try to ping your nameserver (configured in /etc/resolv.conf)
5. Check if your DNS respond to queries try to resolve some hostname using ping google.com or host google.com
 
VladiBG, I'm not sure how you came to that conclusion, but I think you're on to something. I disabled IPv6 in /etc/rc.conf and rebooted five times. Every time networking functioned properly. I then re-enabled IPv6, rebooted and the problem came back, taking out both IPv6 and IPv4.

The results of your suggested steps are:
1. The IPv4 address and netmask for re0 look good, and re0 status is active.
2. I am unable to ping the default gateway IP address, either IPv4 or IPv6.
3. I am unable to ping either 8.8.8.8 or 2001:4860:4860::8888
4. My DNS (currently 1.1.1.1) is also not reachable, and therefore
5. My DNS is unable to resolve hostnames.

I'm able to get it to work if I issue the following four commands:
Code: 
service netif restart
service routing restart
service netif restart
service routing restart
Yes, the same two commands done twice. Before I issue these commands, doing a ping results in nothing:
Code: 
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
$ ping 2001:4860:4860::8888
PING6(56=40+8+8 bytes) 2605:a601:afeb:d901:5a11:22ff:feb8:ea54 --> 2001:4860:4860::8888
^C
--- 2001:4860:4860::8888 ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
After issuing those four commands, ping via IPv4 works, but not via IPv6.
Code: 
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=57 time=13.072 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=12.512 ms
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
$ ping6 2001:4860:4860::8888
ping6: UDP connect: No route to host

Side note: as I dug further, I learned ntpdate is on its way out so I've switched to ntpd. Even if networking isn't working, it at least doesn't hang for 3 minutes at boot time like ntpdate did.
 
JT42 said:
VladiBG, I'm not sure how you came to that conclusion
Click to expand...
From the error of unsupported address family during the resolving of the ntp hostname.

For the test try to comment out the ipv6 line in your /etc/rc.conf file #ifconfig_re0_ipv6="inet6 accept_rtadv" then reboot and verify the connectivity to your gateway with ping 192.168.1.1 if you have connection to your router at 192.168.1.1 then try next step and check if you can reach the DNS at 1.1.1.1 or your ISP provided DNS.

If the IPv4 is working (all of above) then uncomment the IPv6 in your /etc/rc.conf and disable IPv4 address of your interface so you can leave only IPv6 enabled, reboot and try again same test but this time with your IPv6 address of your router and ipv6 address of the DNS server.

This way you will know if the IPv4 is working and if the IPv6 is working. If you have another device with IPv6 you can verify from it if your routers ipv6 have connection to outside world and you can also try to login into your router and verify it's ipv6 settings.
 
given this is a realtek chipset I'd start with disabling various hardware offloading (LRO, RC/TXSUM, VLAN_HWTAGGING...). Wouldn't be the first realtek chipset where offloading is broken and causes problems...
 
VladiBG:
If I enable only ipv4, then ipv4 works.
If I enable only ipv6, then ipv6 does not work
If I enable both, neither works until I issue service netif restart and service routing restart (sometimes I have to do both of those twice), at which point ipv4 works but ipv6 does not.

I can confirm ipv6 is working at my router by using a different device on this network -- I'm able to ping 2606:4700:4700::1111 from that device.

pi@:
It's already installed. I forced a re-installation but it did not help.

sko:
I'll try that next. For what it's worth, this same hardware worked just fine with 13.2 -- net networking problem only started when I switched to 14.0.
 
You can change the ip6addrctl_policy to prefer ipv4 (ipv4_prefer) but this will only make a workaround of the problem and it will not actually fix your ipv6 issue. You need to test why your ipv6 is not reachable from the router. Until then you can use ipv4 only until you find the issue with the ipv6.
 
sko:
I rebooted the machine so it's in the broken state. I started a ping command in one Xterm and then used a second Xterm to start changing the re0 interface with commands such as ifconfig re0 -lro, ifconfig re0 -rxcsum, ifconfig re0 -txcsum, etc..

When I turned off -rxcsum a few ping responses would come through and then it would stop again. It did this for each of the following: -rxcsum -rxcsum6 -txcsum -txcsum6. I turned the rest of them off, too, such that the only option left was VLAN_MTU.

Annoyingly, I did this twice -- the first time the interface started working and I was able to reliably ping both ipv5 and ipv6. But the second time I did it, even with all the options removed (except VLAN_MTU), it didn't work until I did service netif restart and service routing restart, at which point ipv4 worked but ipv6 still did not.
 
Just to follow up on this: I was seeing some other strangeness after this upgrade such as not being able to switch back to any of the text consoles when X was running (but only when launched through xdm) and random characters appearing in the text consoles after X was started. I decided to do a full backup and then a fresh install of 14.0, and it appears to have cleared everything up. I still need to check out a few other things, but so far so good. This solved the two strange things I mentioned above, plus both IPv4 and IPv6 networking appear to be functioning normally.
 
Well, I spoke too soon. All seemed to be going well but I just noticed the random-characters-at-the-console problem is back and I'm no longer able to switch to the text consoles from X. Looking at /var/log/messages it appears the problem started about two days ago, but I don't know exactly what I changed around that time. So far networking, both IPv4 and IPv6 are still functional.
 
You must log in or register to reply here.
Back
Top