Internet works, but ping does not.

This is FreeBSD forum and questions about other OS won't get any answers.
VladiBG Thanks # Understood. I mentioned it in the context of a freebsd VM not connecting to another which happens to a VM residing under another O/S, due to the possibility that the erorrs might be at either of the ends. There was one response that said it was difficult to help without understanding the network topography which in this case is far too simple, so I outlined it. In any case I will confine to the freebsd issues.
 
You start to mix different problems into one forum thread so lets go back to your original problem. To test your network, disable any firewall that you have, setup your re0 interface to obtain the IP address from your broadband router and test if you have ping to the outside. After you have good working internet connectivity then you can add complexity by creating bhyve VM and adding bridges and firewall. So take one step at a time.
 
Thank you VladiBG All of the following are issues with the freebsd bhyve. These are my rc.conf settings, variously changed:

/etc/rc.conf # freebsd native machine pertinent entries # updated, as of now:
sddm_enable="YES"
syslogd_flags="-ss"
dumpdev="NO"
nfs_reserved_port_only="NO"
#dhcpd_enable="YES"
#dhcpd_ifaces="re0 re0bridge1"
cloned_interfaces="bridge1 tap0"
ifconfig_bridge1_name="re0bridge1"
ifconfig_re0="inet 192.168.1.130 netmask 255.255.255.0 up"
#ifconfig_re0="DHCP ether 3c:7c:3f:0f:b6:5f"
ifconfig_re0bridge1="inet 192.168.1.3 netmask 255.255.255.0 addm re0 addm tap0 up"
# byhve and jail settings bridges
gateway_enable="YES"
defaultrouter="192.168.1.1" (this has been 192.168.1.1 but the vm did not reflect this)
#defaultroute_delay="1" # Don't wait for a default route in the foreground
vm_enable="YES"
vm_dir="zfs:zroot/bhyvedata"
#bhyve.svm_modinit="YES"

ifconfig #bsd native machine

re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether 3c:7c:3f:0f:b6:5f
inet 192.168.1.130 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
re0bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:ff:cd
inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 55
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 58:9c:fc:10:22:63
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Internet works. Ping still does not.

bhyve start up script that I use

ifconfig bridge1 create
ifconfig bridge1 addm re0
ifconfig bridge1 name re0bridge1
ifconfig re0bridge1 up
####ifconfig re0bridge addm tap0
ifconfig tap0 create
ifconfig re0bridge1 addm tap0
#ifconfig bridge1 addm tap0

if ! kldstat | grep -w vmm.ko
then
kldload -v vmm
fi
if ! kldstat | grep -w nmdm.ko
then
kldload -v nmdm
fi
bhyve -c 1 -m 1G -w -H \
-s 0,hostbridge \
-s 4,virtio-blk,/dev/zvol/zroot/ubuntuvm \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
ubuntuvm

/etc/neplan/01-netcfg.yaml # yaml file of the VM inside

ethernets:
enp0s5 # the interface name that shows as ip a command output in the vm inside.
...
dhcp4: no
addresses: [192.168.1.30/24]
nameservers:
addresses: [8.8.8.8, 1.1.1.1]
...

VM gets the IP as specified, but there is no connectivity. Ping does not work, doesn't fetch updates.

Update: VM works fine. ip route add default via 192.168.1.1 brought the Internet back to the VM. This is a temporary fix, but I managed to ping from the bhyve VM another machine, and send a message, which was the challenge for the last one week.

On the (native) freebsd machine this is what shows:

netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 192.168.1.1 UGS re0
127.0.0.1 link#2 UHS lo0
192.168.1.0/24 link#1 U re0
192.168.1.3 link#3 UHS lo0
192.168.1.130 link#1 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#2 UHS lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 ::1 UGRS lo0
arp -an
? (192.168.1.3) at 58:9c:fc:10:ff:cd on re0bridge1 permanent [bridge]
? (192.168.1.1) at 00:6d:61:ac:ea:2f on re0 expires in 818 seconds [ethernet]
? (192.168.1.130) at 3c:7c:3f:0f:b6:5f on re0 permanent [ethernet]
 
Hm, you could try to look at network traffic using wireshark, and see what is the difference in the successful / failed attempts. Wondering aloud, can it be IPv6 vs IPv4? Could try to ping passing -4 or -6 to the host or the VM (while targeting a domain name with ping).
In the new FreeBSD 13.1 you have to explicitly tell the ping command to use ipv4 or ipv6
ping -4 -c 3 he.net
ping -6 -c 3 he.net

ping -c 3 he.net I believe defaults to ipv6 and fails when pinging an ipv4 address. I will read through the other 2 pages of prolific testing results and see if you came to the same conclusion.

examples:
ping -4 -c 3 he.net
PING he.net (216.218.236.2): 56 data bytes
64 bytes from 216.218.236.2: icmp_seq=0 ttl=51 time=33.104 ms
64 bytes from 216.218.236.2: icmp_seq=1 ttl=51 time=38.179 ms
64 bytes from 216.218.236.2: icmp_seq=2 ttl=51 time=37.070 ms

--- he.net ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 33.104/36.118/38.179/2.178 ms
ping -6 -c 3 he.net
ping: UDP connect: No route to host
ping -6 -c 3 8.8.4.4
ping: IPv6 requested but IPv4 target address provided

ping -4 -c 3 8.8.4.4
PING 8.8.4.4 (8.8.4.4): 56 data bytes
64 bytes from 8.8.4.4: icmp_seq=0 ttl=113 time=21.249 ms
64 bytes from 8.8.4.4: icmp_seq=1 ttl=113 time=15.719 ms
64 bytes from 8.8.4.4: icmp_seq=2 ttl=113 time=17.307 ms

--- 8.8.4.4 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 15.719/18.092/21.249/2.325 ms

Is 1.1 1.1 IPv4 or IPv6?


Replace those addresses with the 1.1.1.1 DNS addresses: For IPv4: 1.1.1.1 and 1.0.0.1. For IPv6: 2606:4700:4700::1111 and 2606:4700:4700::1001.

ping -6 -c 3 2606:4700:4700::1001
ping: UDP connect: No route to host
ping -4 -c 3 2606:4700:4700::1001
ping: IPv4 requested but IPv6 target address provided

so, I don't have IPv6 networking setup.

Testing ARP
arp -an
? (192.168.1.1) at 00:01:36:1f:e4:59 on wlan1 expires in 1169 seconds [ethernet]
? (192.168.1.37) at 7c:5c:f8:68:5a:c9 on wlan1 permanent [ethernet]
? (192.168.1.48) at 14:bb:6e:e1:72:62 on wlan1 expires in 1197 seconds [ethernet]

Testing netstat:
netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 192.168.1.1 UGS wlan1
127.0.0.1 link#2 UH lo0
192.168.1.0/24 link#3 U wlan1
192.168.1.37 link#3 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#2 UHS lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 ::1 UGRS lo0


I see dhcp4: no
this must be why you manually have to add a route to the gateway 192.168.1.1 for your re0bridge
 
Thank you wb7odyfred

In the new FreeBSD 13.1 you have to explicitly tell the ping command to use ipv4 or ipv6
ping -4 -c 3 he.net
(When Internet is connected and the browser browses)
3 packets transmitted, 0 packets received, 100% packet loss

ping -6 -c 3 he.net
ping: UDP connect: No route to host.
(My ISP railnet hasn't adopted and implemented ipv6)

arp -an
netstat -rn

arp -rn #screenshot attached.
Screenshot_20220919_171238.png

? (192.168.1.1) at 00:6d:61:ac:ea:2f on re0 expires in 1177 seconds [ethernet]
? (192.168.1.130) at 3c:7c:3f:0f:b6:5f on re0 permanent [ethernet]
# netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 192.168.1.1 UGS re0
127.0.0.1 link#2 UH lo0
192.168.1.0/24 link#1 U re0
192.168.1.130 link#1 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#2 UHS lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 ::1 UGRS lo0
I see dhcp4: no
updated rc.conf
#dhcpd_enable="YES"
#dhcpd_ifaces="re0 re0bridge1"
cloned_interfaces="bridge1 tap0"
ifconfig_bridge1_name="re0bridge1"
ifconfig_re0="inet 192.168.1.130 netmask 255.255.255.0 up"
#ifconfig_re0="DHCP ether 3c:7c:3f:0f:b6:5f"
ifconfig_re0bridge1="inet 192.168.1.3 netmask 255.255.255.0 addm re0 addm tap0 up"
#ifconfig_re0bridge1="addm re0 addm tap0 up"
# byhve and jail settings bridges
routed_enable="YES"
defaultrouter="192.168.1.1"
defaultroute_delay="1" # Don't wait for a default route in the foreground
#ignore ICMP redirect="YES"
named_enable="YES"

this must be why you manually have to add a route to the gateway 192.168.1.1 for your re0bridge
I have set the IP address as 192.168.1.130, router's IP remains as 192.168.1.1. Should I set link#3 address also as 192.168.1.130/24 ?

VladiBG

ip6addrctl show
Prefix Prec Label Use
::1/128 50 0 0
::/0 40 1 0
::ffff:0.0.0.0/96 100 4 0
2002::/16 30 2 0
2001::/32 5 5 0
fc00::/7 3 13 0
::/96 1 3 0
fec0::/10 1 11 0
3ffe::/16 1 12 0

service routing restart
/etc/rc.conf: =YES: not found
/etc/rc.conf: ignore: not found
/etc/rc.conf: =YES: not found
/etc/rc.conf: ignore: not found
delete host 127.0.0.1: gateway lo0
delete net default: gateway 192.168.1.1
delete host ::1: gateway lo0
delete net fe80::: gateway ::1
delete net ff02::: gateway ::1
delete net ::ffff:0.0.0.0: gateway ::1
delete net ::0.0.0.0: gateway ::1
add host 127.0.0.1: gateway lo0
add net default: gateway 192.168.1.1
Additional inet routing options: ignore ICMP redirect=YES.
add host ::1: gateway lo0
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1

# ipv6 is not enabled by the ISP

Thank you.
 
Last edited:
As a further update, it works when I ping an IP from the router i works, when I ping from the bhyve VM it works, but from the freebsd base machine terminal, ping does not work. However from the base machine terminal, curl works, traceroute works.

curl icanhazip.com
shows my static IP
ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
^X^C
--- 1.1.1.1 ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss
# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 40 byte packets
1 192.168.1.1 (192.168.1.1) 0.450 ms 0.319 ms 0.237 ms
2 100.108.0.2 (100.108.0.2) 0.991 ms * *
3 * 169.254.0.10 (169.254.0.10) 2.071 ms *
4 * * *
5 * * 172.31.200.95 (172.31.200.95) 5.923 ms
6 * * *
7 * * *
8 172.31.2.63 (172.31.2.63) 17.082 ms * *
9 * * *
10 * * *
11 * * *
12 dns.google (8.8.8.8) 12.922 ms * *
 
Back
Top