I suspect my issue better belongs here than Emulation and Virtualization.
So I'm trying to get a Home-Assistant VM in bhyve (via vm-bhyve) to have a general LAN + Internet, plus IoT/no-internet VLAN to segment off that untrusted crapola that is a lot of the (although useful) IoT. I've got opnSense on the router, and some Ubiquiti switches. Router & FreeBSD NAS/host are on trunk ports. When I plug my laptop into a switch port I set to the VLAN (30) it picks up an IP just fine from the router as I'd expect, within the range I'd expect.
When I set a static IP in the proper range + gateway on the guest, I can ping back and forth fine.
When I try DHCP, using
Can't quite figure out what is wrong. Seems like it should be firewall, but then again my laptop pulls DHCP just fine. Do I need some additional firewall rules for the host VLAN interface that vm-bhyve created?
My configuration of the vm-bhyve switch set with the VLAN
`ha-iot` is the switch name, public is the standard one for all the VMs (up until now the only VMs) that also need Internet access and full LAN access.
The VM configuration
The host interfaces
And finally, the `ipfw` configuration, which I think is wide open
So I'm trying to get a Home-Assistant VM in bhyve (via vm-bhyve) to have a general LAN + Internet, plus IoT/no-internet VLAN to segment off that untrusted crapola that is a lot of the (although useful) IoT. I've got opnSense on the router, and some Ubiquiti switches. Router & FreeBSD NAS/host are on trunk ports. When I plug my laptop into a switch port I set to the VLAN (30) it picks up an IP just fine from the router as I'd expect, within the range I'd expect.
When I set a static IP in the proper range + gateway on the guest, I can ping back and forth fine.
When I try DHCP, using
tcpdump -i em0.30 port 67 or port 68 -e -n -vv on the host, I see the DHCP request, but no DHCP offer. On the router on the VLAN30 interface, I see both DHCP request, and DHCP offer. Can't quite figure out what is wrong. Seems like it should be firewall, but then again my laptop pulls DHCP just fine. Do I need some additional firewall rules for the host VLAN interface that vm-bhyve created?
My configuration of the vm-bhyve switch set with the VLAN
Code:
# vm switch list
NAME TYPE IFACE ADDRESS PRIVATE MTU VLAN PORTS
public standard vm-public - no - - em0
ha-iot standard vm-ha-iot - no - 30 em0`ha-iot` is the switch name, public is the standard one for all the VMs (up until now the only VMs) that also need Internet access and full LAN access.
The VM configuration
Code:
loader="uefi"
graphics="yes"
xhci_mouse="yes"
graphics_listen="192.168.2.5"
graphics_port="5900"
graphics_wait="no"
graphics_res="800x600"
cpu="4"
memory="4GB"
network0_type="virtio-net"
network0_switch="public"
disk0_type="ahci-hd"
disk0_name="disk0"
disk0_dev="sparse-zvol"
uuid="9edcc9da-1b35-11eb-8576-0015170027d2"
network0_mac="58:9c:fc:06:f4:a0"
network1_type="virtio-net"
network1_switch="ha-iot"
network1_mac="58:9c:fc:5f:71:50"The host interfaces
Code:
# ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=812099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
ether 00:15:17:00:27:d2
inet 192.168.2.5 netmask 0xffffff00 broadcast 192.168.2.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether 70:85:c2:fc:1f:58
media: Ethernet autoselect (none)
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo1: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
groups: lo
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 2e:36:b0:9f:36:7f
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 10 priority 128 path cost 2000000
member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 9 priority 128 path cost 2000000
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 8 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge vm-switch viid-4c918@
nd6 options=1<PERFORMNUD>
vm-ha-iot: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 6e:19:b8:bb:03:25
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 11 priority 128 path cost 2000000
member: em0.30 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 7 priority 128 path cost 55
groups: bridge vm-switch viid-f27b7@
nd6 options=1<PERFORMNUD>
em0.30: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vm-vlan-ha-iot-em0.30
options=1<RXCSUM>
ether 00:15:17:00:27:d2
inet6 fe80::215:17ff:fe00:27d2%em0.30 prefixlen 64 scopeid 0x7
groups: vlan vm-vlan viid-ccc4e@
vlan: 30 vlanpcp: 0 parent interface: em0
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vmnet-unifi-controller-0-public
options=80000<LINKSTATE>
ether 58:9c:fc:10:ff:87
groups: tap vm-port
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 2153
tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vmnet-pihole-0-public
options=80000<LINKSTATE>
ether 58:9c:fc:10:ff:a9
inet6 fe80::5a9c:fcff:fe10:ffa9%tap1 prefixlen 64 tentative scopeid 0x9
groups: tap vm-port
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 2454
tap2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vmnet-home-assistant-0-public
options=80000<LINKSTATE>
ether 58:9c:fc:10:f2:21
inet6 fe80::5a9c:fcff:fe10:f221%tap2 prefixlen 64 tentative scopeid 0xa
groups: tap vm-port
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 2385
tap3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vmnet-home-assistant-1-ha-iot
options=80000<LINKSTATE>
ether 58:9c:fc:10:60:08
inet6 fe80::5a9c:fcff:fe10:6008%tap3 prefixlen 64 scopeid 0xb
groups: tap vm-port
media: Ethernet autoselect
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Opened by PID 2385And finally, the `ipfw` configuration, which I think is wide open
Code:
# ipfw list
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any icmp6types 1
01000 allow ipv6-icmp from any to any icmp6types 2,135,136
65000 allow ip from any to any
65535 deny ip from any to any