Solved How to duplicate the tap0 interface in a new tap1 network interface that it is not used

ok i manage to reproduce your problem. When you start your VM using the .sh script you have "&" at the end of the bhyve command and when you destroy the VM using bhyvectl --vm=vm0 --destroy the bhyve process continue to run which can be observed using ps aux | grep bhyve and also in top

last pid: 1432; load averages: 0.36, 0.17, 0.08; battery: 98% up 0+00:10:30 20:35:38
33 processes: 1 running, 31 sleeping, 1 stopped
CPU: 0.1% user, 0.0% nice, 0.5% system, 0.0% interrupt, 99.4% idle
Mem: 76M Active, 5064K Inact, 284M Wired, 124M Buf, 3315M Free
Swap: 4096M Total, 4096M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
1286 root 11 20 0 148M 48M select 3 0:01 0.00% Xorg
1276 root 1 52 0 44M 24M select 0 0:00 0.00% slim
1343 root 1 20 0 16M 4860K ttyin 2 0:00 0.00% csh
1336 versus 1 20 0 21M 9092K select 1 0:00 0.00% sshd
1393 root 1 20 0 16M 4600K pause 2 0:00 0.00% csh
1386 versus 1 20 0 21M 9132K select 2 0:00 0.00% sshd
1400 root 20 22 0 4174M 29M STOP 2 0:00 0.00% bhyve
1333 root 1 21 0 21M 9060K select 2 0:00 0.00% sshd
1383 root 1 21 0 21M 9096K select 2 0:00 0.00% sshd
1298 root 1 20 0 18M 6880K select 0 0:00 0.00% sendmail
1337 versus 1 24 0 14M 3808K pause 2 0:00 0.00% csh
1171 root 1 20 0 13M 2632K select 2 0:00 0.00% syslogd
1387 versus 1 24 0 14M 3860K pause 2 0:00 0.00% csh
977 root 1 20 0 11M 1432K select 0 0:00 0.00% devd
1392 versus 1 22 0 13M 3176K wait 2 0:00 0.00% su
1342 versus 1 21 0 13M 3176K wait 3 0:00 0.00% su
1305 root 1 20 0 13M 2628K nanslp 0 0:00 0.00% cron
1295 root 1 20 0 20M 7940K select 1 0:00 0.00% sshd
1323 root 1 52 0 13M 2268K ttyin 3 0:00 0.00% getty
1329 root 1 52 0 13M 2268K ttyin 1 0:00 0.00% getty
1324 root 1 52 0 13M 2268K ttyin 2 0:00 0.00% getty
1327 root 1 52 0 13M 2268K ttyin 3 0:00 0.00% getty
1328 root 1 52 0 13M 2268K ttyin 3 0:00 0.00% getty
1325 root 1 52 0 13M 2268K ttyin 0 0:00 0.00% getty
1326 root 1 52 0 13M 2268K ttyin 1 0:00 0.00% getty
1330 root 1 52 0 13M 2268K ttyin 0 0:00 0.00% getty
1301 smmsp 1 52 0 18M 6340K pause 1 0:00 0.00% sendmail
807 root 1 52 0 13M 2612K select 0 0:00 0.00% dhclient
873 _dhcp 1 20 0 13M 2820K select 1 0:00 0.00% dhclient
810 root 1 4 0 13M 2716K select 1 0:00 0.00% dhclient
1432 root 1 20 0 14M 3372K CPU1 1 0:00 0.00% top
1252 messagebus 1 52 0 14M 3468K select 1 0:00 0.00% dbus-daemon
1219 _sndio 1 52 -20 14M 2436K select 1 0:00 0.00% sndiod
Click to expand...

root@versus-laptop:/dev # ps aux | grep bhyve
root 1400 0.0 0.8 4274176 29976 0 T 20:31 0:00.02 bhyve -c 2 -m 4G -w -H -s 0,hostbridge -s 3,ahci-cd,/home/versus/mini.iso -s 4,ahci-hd,guest.img -s 5,virtio
root 1434 0.0 0.1 12868 2348 0 S+ 20:36 0:00.00 grep bhyve
Click to expand...

but vmm device under /dev/vmm/vm0 is removed and the tap0 interface is keep busy then if you try to start a new vm it will fail with error

open of tap device /dev/tap0 failed
device emulation initialization error: Device busy
Click to expand...

And even after kill your bhyve process and start it again with the same script which try to create additional tap0 and bridge0 which already exist it will start the VM and produce the following errors:

Error return from kevent change: No such file or directory
rdmsr to register 0x140 on vcpu 0
wrmsr to register 0x140(0) on vcpu 0
rdmsr to register 0x140 on vcpu 1
wrmsr to register 0x140(0) on vcpu 1
rdmsr to register 0x34 on vcpu 0
Error return from kevent change: No such file or directory
Error return from kevent change: No such file or directory
Error return from kevent change: No such file or directory
Click to expand...

To fix this first add shebang #!/bin/sh to your .sh script and remove the commands which create bridge and tap interface from the script and put them in /etc/rc.conf instead then remove the "&" at the end of bhyve command line which cause your problem with the missing connection.

After you install the VM you can remove the "-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900,wait \" otherwise it will keep waiting for initial connection via vnc and will not start.

You don't need if_bridge_load="YES" and if_tap_load="YES" in your /etc/rc.conf as those drivers are part of the FreeBSD kernel from version 4.1 and they don't need to be loaded as modules. Also in the new version (r347241) tap is replaced by tuntap driver.

You don't need gateway_enable="YES" inside /etc/rc.conf as you are using bridging(L2 switching) not routing between interfaces and you can also remove the net.inet.ip.forwarding=1 from /etc/sysctl.conf as it's those the same as gateway_enable="YES".
 
VladiBG said:
ok i manage to reproduce your problem. When you start your VM using the .sh script you have "&" at the end of the bhyve command and when you destroy the VM using bhyvectl --vm=vm0 --destroy the bhyve process continue to run which can be observed using ps aux | grep bhyve and also in top





but vmm device under /dev/vmm/vm0 is removed and the tap0 interface is keep busy then if you try to start a new vm it will fail with error



To fix this first add shebang #!/bin/sh to your .sh script and remove the commands which create bridge and tap interface from the script and put them in /etc/rc.conf instead Also remove the "&" at the end of bhyve command line.

After you install the VM you can remove the "-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900,wait \" otherwise it will keep waiting for initial connection via vnc and will not start.

You don't need if_bridge_load="YES" and if_tap_load="YES" in your /etc/rc.conf as those drivers are part of the FreeBSD kernel from version 4.1 and they don't need to be loaded as modules. Also in the new version (r347241) tap is replaced by tuntap driver.

You don't need gateway_enable="YES" inside /etc/rc.conf as you are using bridging(L2 switching) not routing between interfaces and you can also remove the net.inet.ip.forwarding=1 from /etc/sysctl.conf as it's those the same as gateway_enable="YES".
Click to expand...

if_tap_load="YES" and if_bridge_load="YES" were on /boot/loader.conf : removed
 
No you don't need gateway_enable and you don't need net.inet.ip.forwarding=1 (which is the actual sysctl) as they are required only if you are routing traffic between interfaces. You are using bridge0 which act as L2 switch and your VM became part of your internal network as they are connected to your home router.
 
ziomario said:
if_tap_load="YES" and if_bridge_load="YES" were on /boot/loader.conf : removed
Click to expand...
root@marietto:/home/marietto # virt-host-validate bhyve

BHYVE: Checking for vmm module : PASS
BHYVE: Checking for if_tap module : WARN (if_tap module is not loaded, networking will not work)
BHYVE: Checking for if_bridge module : PASS
BHYVE: Checking for nmdm module : PASS
 
Not sure why this is marked Solved.

The OP's original thesis was correct. You assign a separate tap interface to each vm. The taps are each linked together by a bridge interface. The Internet gateway is routed to the bridge interface. You thereby can access the gateway from each vm.

Code: 
ifconfig tap0 create
ifconfig tap1 create
ifconfig bridge0 create
ifconfig bridge0 addm tap0 addm tap1
ifconfig bridge0 up
ifconfig bridge0 inet <gateway> netmask <mask>

The second part is configuring the vm network config itself.

The vm would have an interface address on the gateway's subnet and the route is the route on the BSD host.
 
You must log in or register to reply here.
Back
Top