bhyve How to connect two bhyve vms on internet using two tap interfaces...

ziomario

Aspiring Daemon

Reaction score: 42
Messages: 608

Hello to everyone.

I would like to use two bhyve virtual machines at the same time because I want to test if the gpu passthrough is working on the Windows 10 vm. To do that,I need to disable the framebuffer on the Win10 vm,because,probably the nvidia driver does not work if it is enabled. When it is disabled I need also to see what's happening inside,specially if the nvidia driver produces the error 43. For this reason I've installed teamviewer client on the Ubuntu vm and teamviewer server on Windows 10 vm. Unfortunately I can't use the same tap0 interface inside two virtual machines running at the same time. To fix this issue I've modified the rc.conf file like below :


Code:
gedit /etc/rc.conf :

cloned_interfaces="bridge0 tap0 bridge1 tap1"
ifconfig_bridge0="addm em0 addm tap0"
ifconfig_bridge1="addm em0 addm tap1"

With those parameters configured my network configuration looks like :

Code:
# ifconfig

em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d8:5e:e2:1f:23
inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 51:9c:fc:10:ff:b7
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 2000000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>

tap0: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 51:9c:fc:10:ff:8e
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 51:9c:fc:10:65:6c
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>

tap1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 51:9c:fc:10:ff:e9
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


Below u can see how I have configured the bhyve parameters for both the vms :


gedit os-uefi-ub2104.sh :


Code:
bhyve -S -c 4 -m 8G -w -H \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Downloads/OS/ubuntu-21.04-desktop-amd64.iso \
-s 2,virtio-blk,/mnt/da3s1/OS/bhyve/ub2104.img \
-s 8,virtio-net,tap0 \
-s 30,xhci,tablet \
-s 31,lpc \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
vm0


gedit os-uefi-win10.sh :


Code:
bhyve -S -c 4 -m 8G -w -H \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/da2p1/vms/os/windows10/windows10.img \
-s 2,passthru,1/0/0 \
-s 3,passthru,1/0/1 \
-s 4,passthru,1/0/2 \
-s 5,passthru,1/0/3 \
-s 8,virtio-net,tap1 \
#-s 29,fbuf,tcp=0.0.0.0:5901,w=1440,h=900,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
vm1

now the problem I have is that the Windows 10 vm can't connect to internet. Can someone explains why ? thanks.
 

rootbert

Well-Known Member

Reaction score: 169
Messages: 427

Why don't you use one bridge? - Does network work if you try to connect two linux machines? Are the bridge/tap interfaces "up"? Did you install virtio drivers inside Windows? did you set net.link.tap.up_on_open? Is packet forwarding enabled on the bridges?
 
OP
ziomario

ziomario

Aspiring Daemon

Reaction score: 42
Messages: 608

Why don't you use one bridge? - Does network work if you try to connect two linux machines? Are the bridge/tap interfaces "up"? Did you install virtio drivers inside Windows? did you set net.link.tap.up_on_open? Is packet forwarding enabled on the bridges?

is this correct ?

Code:
cloned_interfaces="bridge0 tap0 tap1"      
ifconfig_bridge0="addm em0 addm tap0 addm tap1"

I see that the problem on the bridge1 is that it does not include these arguments :

Code:
member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>

so,it is not well configured.
 

rootbert

Well-Known Member

Reaction score: 169
Messages: 427

Code:
ifconfig_tap0="up"
ifconfig_tap1="up"
ifconfig_tap2="up"
ifconfig_epair1a="up"
ifconfig_epair2a="up"
ifconfig_bge1="up"
ifconfig_bge2="up"

ifconfig_bridge0="addm tap0 addm tap1 \
addm epair1a addm epair2a \
addm bge1 up"

... an example from one of my workstations (your cloned_interfaces is ok ...)
 
OP
ziomario

ziomario

Aspiring Daemon

Reaction score: 42
Messages: 608

I've checked. this is the solution that worked :

Code:
cloned_interfaces="bridge0 tap0 tap1"    
ifconfig_bridge0="addm em0 addm tap0 addm tap1"

so,thanks,when you have suggested to use one bridge you really helped me.
 
Top