Internet works, but ping does not.

These two need discussing too.

gateway_enable="YES"
defaultrouter="192.168.1.13"

Really odd that your gateway IP is the same as the default router.
Typically your fiber router would be 192.168.1.1 and gateway 192.168.1.255.

This 192.168.1.130 box is not really acting as the main gateway here. The fiber router is.
So try axing this setting. Default router should probably be 192.168.1.1.

Consider commenting both of these out when fixing dhcp server.
 
Really odd that your gateway IP is the same as the default router.
Typically your fiber router would be 192.168.1.1 and gateway 192.168.1.255.
.255 is the broadcast address of 192.168.1.0/24 it can't be gw. Without knowing the entire network topology is hard to make any assumptions.
 
Phishfry I set up the router ip in the router as 192.168.1.13 and the router login page shows only when I type http://192.168.1.13/admin/login_en.asp in the browser bar, not when I type 192.168.1.1. I have configured it as such. If the gateway is different from the router IP, what do I need to say in /etc/rc.conf ?

VladiBG I have one wired broadband connection with a router IP 192.168.1.13, with two wires out, one going to the freebsd machine set to 192.168.1.130, another to the ubuntu machine for which I set the computer's interface IP in the router as 192.168.1.131 (but this ubuntu machine gets a dynamic address). Each of these machines have a VM running inside, freebsd vm gets almost a static VM IP 1.2, but the VM inside Ubuntu gets a dynamic IP. All four private IPs show up in curl icanhazip.com as one public IP. Attaching the ifconfig screenshot of the freebsd machine.

Thank you
 

Attachments

  • ubuntu ip a screenshot.png
    ubuntu ip a screenshot.png
    180.2 KB · Views: 18
  • ifconfig freebsd.png
    ifconfig freebsd.png
    180.7 KB · Views: 18
If you have DHCP server on your ubuntu you should disabled it and use only the DHCP server on your broadband router.
If there's no other device on 192.168.1.1 i would recommend to change the IP address of your broadband router from 192.168.1.13 to 192.168.1.1 but first check if there's another device by using ping 192.168.1.1 and then check the arp table with arp 192.168.1.1 if there's MAC address for 192.168.1.1 then you have another device with this IP.
This network configuration is not related in any way to your initial problem.
 
Thank you VladiBG.

Base machine 1 (freebsd) physical interface re0 doesn't show IP. Bridge name re0bridge1; ifconfig shows IP address (locally set static) as set 192.168.1.130. Ping doesn't work Internet works.
arp 192.168.1.1 shows no entry
arp 192.168.1.13 shows 00:6d:61:ac:ea:2f on re0bridge1
VM shows connected interface name as enp0s5 shows IP as 192.168.1.2
arp 192.168.1.1 from the vm inside shows incomplete enp0s5,
arp192.168.1.13 shows 00:6d:61:ac:ea:2f flag C enp0s5 Ping works Internet works

Base machine 2 (ubuntu) physical interface enp6s4f0 doesn't show IP; Bridge name virbr0 shows dynamic IP 192.168.1.12
arp 192.168.1.1 shows incomplete virbr0
arp192.168.1.13 shows 00:6d:61:ac:ea:2f virbr0
VM shows connected interface name as enp1s0 shows IP as 192.168.1.11
arp 192.168.1.1 from the vm inside shows incomplete enp1s0
arp192.168.1.13 shows 00:6d:61:ac:ea:2f flag C enp1s0

ping 192.168.1.1 from base machines and the vms shows host unreachable
ping 192.168.1.13 works from all machines (except in freebsd machine in which ping doesn't work in the native, but works in the vm inside)

Thank you.
 
Then if there's no other specific reason of using 192.168.1.13 on your router you can change it's address to 192.168.1.1 and check it's DHCP scope settings. I recommend to configure the dhcp scope to lease the addresses from 192.168.1.50 - 192.168.1.200, and use the low range addresses from 192.168.1.2 - 192.168.1.49 for static IP addresses for WIFI AP, Switches, Servers, and so on. This way there will be no IP address conflicts between the dynamic DHCP range and static.

Did you check if you have DHCP service on your Ubuntu ?
 
So in fact gw and default router should be the same.
You may have several gateways to different networks and more than one default route also know as last resort 0.0.0.0/0 with different metric/weight. For small home network you have only one gateway which in fact is your last resort route aka default route.
 
Then if there's no other specific reason of using 192.168.1.13 on your router you can change it's address to 192.168.1.1 and check it's DHCP scope settings. I recommend to configure the dhcp scope to lease the addresses from 192.168.1.50 - 192.168.1.200, and use the low range addresses from 192.168.1.2 - 192.168.1.49 for static IP addresses for WIFI AP, Switches, Servers, and so on. This way there will be no IP address conflicts between the dynamic DHCP range and static.

Did you check if you have DHCP service on your Ubuntu ?

Thank you. I will reset router IP to 1.1 and find a way to check DHCP scope settings, and configure as you have suggested.

Ubuntu takes time to boot configuring DHCP. I tried setting a static IP using netplan, but the yaml especially with specifications for VMs was so pointless that I deinstalled netplan.io and now it would be easy to configure using /etc/network/interfaces But strangely, even before I have started, when I switched on the Ubuntu machine it shows the static IP as originally set on enp6s4f0 as 192.168.1.131 and virbr0 as a 192.168.1.8 :) Don't know how it happened. I will now go into dhcp settings and disable dhcp completely.

Update: With this ifconfig output on ubuntu there is no Internet and ping does not work. Systemctl does not restart networking. There is a libvirt routine that I need to follow to stop and restart libvirtd and run some vrish commands to enable bridge vrbr0 and then restart networking, in which case the static ip will be gone. Will edit in a while. Thanks.

Thank you.
 
OK so now that I have thought about your post it seems you are running libvirtd VM's under bhyve.
Ubuntu being one of them.
When I ran libvirtd I used thier e1000 driver for networking on libvirtd.
That won't help in this case with Realtek interface in use.
It does look like there is a generic driver:
Virtual Realtek 8139 (rtl8139) NIC

That is what you should be trying. I doubt you need a virbr interface.
 
Looking at your screenshot of Ubuntu in post #28 I see the IP's assigned and I see two IP's attached to virbr0
That looks incorrect.
I do see you have several down interfaces in that VM.
 
It looks like you have 4 network interfaces attached to your Ubuntu VM
enp6s4f0 << UP
enp0s8 << DOWN
enp6s4f1 << DOWN
enp0s9 << DOWN

then bridge virbr0
and vnet0

I would expect one interface like enp0s8 and vnet0.
Do you know what the other interfaces are doing? How did you pass them thru? Via libvirtd configuration?
Virsh or manual xml editing?
Do you have additional network interfaces in your host besides re0 ?

I am not proficient in linux networking so that looks different than my devuan instance.

Your FreeBSD ifconfig looks correct now.
 
OK so now that I have thought about your post it seems you are running libvirtd VM's under bhyve.
Ubuntu being one of them.

Phishfry No. I am not running libvirtd under bhyve. I followed the bhyve installtion process that began with instructions to create a bridge.

The two screenshots in post 28 show the terminal of two native machines, one freebsd and another ubuntu.

When I ran libvirtd I used thier e1000 driver for networking on libvirtd.
That won't help in this case with Realtek interface in use.
It does look like there is a generic driver:
Virtual Realtek 8139 (rtl8139) NIC

That is what you should be trying. I doubt you need a virbr interface.

Thank you for the documentation. I will read the redhat documentation and if there is some way libvirtd in ubuntu will make a ubuntu vm work without a bridge, I will do that.

VladiBG I have reconfigured the router with the IP 192.168.1.1. I have set static and specified the IP address in /etc/network/interfaces. The dhcp settting defaults as typically set by the ISP in customer premises are attached as screenshots. I can modify them suitably, if you could tell me what to change. Thank you
 

Attachments

  • router recommended settings PPPoE not dhcp.png
    router recommended settings PPPoE not dhcp.png
    49.4 KB · Views: 6
  • router dhcp server settings.png
    router dhcp server settings.png
    69.4 KB · Views: 6
But you mentioned virsh to create bridge. So I assumed that was on the FreeBSD host.

So that is what you used under Ubuntu to make a virtual bridge. OK I understand. virsh used on virt client.
That don't sound like correct usage.
Virsh is virtualization host command line tool for libvirtd. Configures libvirt VM's.
 
OK on your "router DHCP webpage" image you want to set aside a range for static IP's.
You have from 192.168.1.2 to 192.168.1.254.
That leaves no free space for static DHCP leases.

Try this instead.
Start IP Address= 192.168.1.100
End IP Address = 192.168.1.254

Then use the button on same page " Edit Reserved IP Address" to setup your Static DHCP IP's"
You will need MAC address for ID.

Here is a big hint. You want to assign a MAC address to your 're0bridge1' in rc.conf.
Then use that MAC address in your router for Static DHCP IP settings.

The reason: Software bridges use no set MAC and should be defined.
 
re0bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 58:9c:fc:10:ff:cd
So the MAC address of the bridge right now is:
58:9c:fc:10:ff:cd
You should use that for the ifconfig setting and in your Routers "Edit Reserved IP Address" entry.
 
I will read the redhat documentation and if there is some way libvirtd in ubuntu will make a ubuntu vm work without a bridge, I will do that.
Just use that document with a grain of salt. It is stinking old but does illustrate the differences (years ago) between Intel e1000 and realtek r8139 libvirt drivers.

You do not need libvirtd at all on a UbuntuVM under Bhyve. Period. It is for the VM hypervisor. Not clients.
Remove it and let Ubuntu do its thing. You do need to have your Bhyve config right for the VM.

How are you starting your bhyve VM's? Manually from command line or with a script or perhaps a tool?
 
It might help here if I explain what libvirt is/does.
This is a Virtual Machine management layer for the HyperVisor.
One part of libvirt is a command prompt called virsh.
It allows you to edit the VM configurations and start and stop VM's.
It does have drivers to facilitate VM networking and disks.

Bhyve has the ability to use libvirt for management of KVM style VM's.
 
Your router config looks good for me. For the test let the FreeBSD to acquire it's IP address via DHCP (for re0 or if you prefer to have ip address on the bridge itself then re0bridge1 BUT not both) and then test if you have ping to your router at 192.168.1.1 and then to your provider gateway at 100.108.0.1

I don't see any reason to have the IP address on the bridge unless you are using routing/forwarding between the bhyve guest VM and the FreeBSD host. When you have bridge between your LAN interface re0 and your bhyve TAP interface then VM is connected on L2 to your broadband router and it will have direct access to it and also can obtain the IP address via DHCP from it. It's not needed to put IP address on your bridge interface in this case.

If you don't bridge your LAN interface with your bhyve TAP interface and if you want your bhyve guest to have access to your network via routing/forwarding then you need to have IP address on the bridge itself which will act as default gateway address for your bhyve guest VM. But then you will need to enable forwarding and use NAT.

So if you don't want to filter/restrict your bhyve VM and let it access the entire local network it's easy to use bridge between your LAN interface re0 and the TAP interface.
 
Thank you Phishfry for the several valuable hints and clear instructions. I changed the IP range to 192.168.1.155 (start) and 254(end) in that page, but after the change took effect, static ips could not be assigned to the interfaces. I had to change back to start at 1.2 and end at 1.254 to make the static ip mapping accepted. ( Earlier, without changing the start and end ip I did assign static ips to the router and computer interfaces, this worked well)

I also changed rc.conf to specify the interface address as you have suggested. Will do the same for ubuntu virbr0.

virsh and libvirtd are in the Ubuntu machine, where I use virt manager; I kept my bhyve installation simple, use KRDC to connect.
 
You don't need DHCP reservation if you are going to use static IP address on your computer.
Only if you want to use DHCP to configure your computer and want to have same IP address every time then you need to create a DHCP reservation within the DHCP scope range.
 
Phishfry Not sure if it was because of ifconfig_re0bridge1="DHCP ether 58:9c:fc:10:ff:cd addm re0 addm tap0" after setting static ips in the router, Internet didn't work in any of the machines, then I tried different changes in rc.conf and tried ifconfig_re0="inet 192.168.1.xxx netmask 255.255.255.0" which reconnected the Internet. This took quite a while and with all the focus on getting the Internet connected, I didn't quite go over the details of fixes needed to get ping work. Will do that tomorrow.

VladiBG I managed to map the interfaces of the base machines, VMs and the bridges to static IPs in the router configuration. Left start and end ips as it was, without reserving a range for dhcp. So far I have checked the ifconfig for the base machines and the two bridges, will have set up the static configurations inside the VMs to map the VM's interfaces to the static IP. It would be good to enable communication between the base machine and the VM, and for this reason, it would be good to assign an IP to the bridge as well. Will follow your other suggestions tomorrow.

Thank you !
 
Your router config looks good for me. For the test let the FreeBSD to acquire it's IP address via DHCP (for re0 or if you prefer to have ip address on the bridge itself then re0bridge1 BUT not both) and then test if you have ping to your router at 192.168.1.1 and then to your provider gateway at 100.108.0.1

I don't see any reason to have the IP address on the bridge unless you are using routing/forwarding between the bhyve guest VM and the FreeBSD host. When you have bridge between your LAN interface re0 and your bhyve TAP interface then VM is connected on L2 to your broadband router and it will have direct access to it and also can obtain the IP address via DHCP from it. It's not needed to put IP address on your bridge interface in this case.

If you don't bridge your LAN interface with your bhyve TAP interface and if you want your bhyve guest to have access to your network via routing/forwarding then you need to have IP address on the bridge itself which will act as default gateway address for your bhyve guest VM. But then you will need to enable forwarding and use NAT.

So if you don't want to filter/restrict your bhyve VM and let it access the entire local network it's easy to use bridge between your LAN interface re0 and the TAP interface.

I had connectivity on both the base Ubuntu machine and in the vm inside, with static IP addresses; I could ssh from the freebsd VM to the ubuntu vm, but all this was short lived.

There was some recurrent trouble when stopping and restarting the ubuntu's vm, which had to routinely dealt with by a series of virsh, brctl and libvirtd commands. Between all this troubleshooting newer problems arose, such as that of the bridge not detected or not working. An easier option is to start afresh with a new installation of the base machine, as some of the network issues relate to the installation without networking, which was set right post-install. (Someone taught me how to set up a linux vm with kimchi, I hadn't taken this familiar route and followed instructions online and had created the ubuntu VM running inside a Ubuntu that innovated network code to feature yaml that counted blank spaces ! )

On both native machines I created ufw rules to allow communication between the VMs, but not quite sure how it reflects on ip tables, because I saw an UFW error in journalctl, which seemed related to rules that I did not create.

All I need to do is the get one machine to ping another machine a feet away, and send a one line message that I need to read in the other machine, but these are the challenges:

1. The physical network interface is mapped to a virtual bridge with a default name as br0 or bridge0, which is changed by libvirtd as virbr0 in ubuntu or assigned a bridge name such as re0bridge1 in freebsd, which in turn has to be interfaced with a virtual interface of the VMs which have different names. Something goes wrong in the process. I need to fix this.

2. The firewall rules have to be just right to allow communication between the VMs, and the IP tables have be right.

These are tasks very ordinary for network experts, due to my non-technical background combined with a non-linear learning path, and due to some inherent issues with the network not really being a network even of a minimal number of computers, this is taking hours after hours everyday for a whole week, not yet resolved. Please guide me on this. Thank you.
 
Top