IPFW bhyve and firewall on host

M

marcinkk

Hi,

I'm testing bhyve.

I moved VM from VirtualBox and successully run it usunig mainly Handbook: https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html and Wiki: https://wiki.freebsd.org/bhyve

Generally works, but I have a question about firewall. To get network working in the guest I had to reconfigure firewall on host:

Code: 
ipfw add 20 allow ip from guest_ip to any
ipfw add 20 allow ip from any to guest_ip

I found that all packet goes by 3 interfaces: tap0 <-> bridge0 <-> lagg0

Is it normal?
How to avaoid filtering these packets?
How to configure firewall to pass DHCP to VM?

At configuration stage I've made:

Code: 
# ifconfig tap0 create
# sysctl net.link.tap.up_on_open=1
net.link.tap.up_on_open: 0 -> 1
# ifconfig bridge0 create
# ifconfig bridge0 addm lagg0 addm tap0
# ifconfig bridge0 up

An then run the VM:

Code: 
grub-bhyve -m ubuntu_server_3.map -r hd0,msdos1 -M 16G ubuntu_server_3

bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./ubuntu_server_3.raw -l com1,/dev/nmdm0A -c 16 -m 16G ubuntu_server_3 &

Now I have on host:

Code: 
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6>
        ether 00:25:90:ca:5e:14
        inet host_ip netmask 0xffffff00 broadcast xxxx
        laggproto failover lagghash l2,l3,l4
        laggport: igb0 flags=5<MASTER,ACTIVE>
        laggport: igb1 flags=0<>
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:f3:1e:f7:00
        groups: tap
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        Opened by PID 75860
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:21:cd:90:56:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 55
        member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 20000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

At guest:

Code: 
enp0s2    Link encap:Ethernet  HWaddr 00:a0:98:24:87:d4
          inet addr:guest_ip  Bcast:xxxx  Mask:255.255.255.0
          inet6 addr: fe80::2a0:98ff:fe24:87d4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:66146 errors:0 dropped:0 overruns:0 frame:0
          TX packets:385 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4015991 (4.0 MB)  TX bytes:45631 (45.6 KB)

Both IPs, host_ip and guest_ip, are in the same subnet.

As I said at the beginning: generally works, but the rules in host firewall are necessary :(
 
I don't know the rest of your firewall rules, but in general you don't need any firewall to enable the guest network when you are using bridge mode
 
marcinkk said:
I found that all packet goes by 3 interfaces: tap0 <-> bridge0 <-> lagg0

Is it normal?
How to avaoid filtering these packets?
How to configure firewall to pass DHCP to VM?
Click to expand...
Well, you bridged tap0 and lagg0. The bridge by definition propagates all packets to the bridged interfaces, so it is not only normal - this is how bridging works.

Regarding the filtering, there is a sysctl variable controlling if packet filtering is applied to bridge interfaces. Take a look here: https://lists.freebsd.org/pipermail/freebsd-ipfw/2014-October/005743.html
and also in the documentation: if_bridge(4)

Check if the following value is set to 0:
Code: 
sysctl net.link.bridge.ipfw
If not, then try setting it in your /boot/loader.conf: net.link.bridge.ipfw=0.
 
roccobaroccoSC said:
(...) and also in the documentation: if_bridge(4)

Check if the following value is set to 0:
Code: 
sysctl net.link.bridge.ipfw
Click to expand...

It was set to 0 but packets were still filtered.

I've set additionally:

Code: 
sysctl net.link.bridge.pfil_bridge=0
sysctl net.link.bridge.pfil_member=0

and now the bhyve machines has network access without additional ipfw rules.

Thanks for hints :)
 
You must log in or register to reply here.
Back
Top