Bridge physical network and bhyve using netgraph not working.

tanis

tanis

Dear @ll,

I'm a bit stuck and would highly appreciate, if someone could help me out or point me in the right direction.

I'm trying to use netgraph to bridge between a bhyve vm and my actual network (LAN). The goal is to get the router's DHCP server assign an IP addr to the bhyve vm (mac: 58:9c:fc:07:7c:be) . I can see the BOOTP/DHCP request arriving on my physical NIC (ix1) attached to the netgraph bridge, but I can't see any of these packets arriving at the router on the actual network.
Code: 
#bhyve host, tcpdump -i ix1
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 58:9c:fc:07:7c:be (oui Unknown), length 300

Code: 
 ------------                              --------
|            |                            |        |
| bhyve host |ix1 ------<LAN>--------- ix1| Router |
|            |                            |  DHCP  |
|            |                            |        |
 ------------                              --------
      |
      |
  -------
 | ngbr0 |
  -------
      |
      |
  -------------
 | bhyve VM    |
 | DHCP Request|
  -------------

My setup:

bhyve hostComments
Code: 
$ ngctl mkpeer ix1: bridge lower link0
Create an unnamed bridge node an attach ix1:lower to unnamed:link0.
Code: 
$ ngctl name ix1:lower ngbr0
Name unamed bridge node to ngbr0.
Code: 
$ ngctl connect ix1: ngbr0: upper link1
Attach ix1:upper to ngbr0:link1 so we can use ix1 as a host interface.
Code: 
$ ngctl msg ix1: setpromisc 1
Enable promiscuous mode, to ensure that the interface will pick up all network packet.
Code: 
$ ngctl msg ix1: setautosrc 0
Don't overwrite the source address on packets.
Code: 
$ vm switch create -t netgraph ngbr0
Configure ngbr0 as virtual switch for bhyve.

Code: 
# bhyve host
$ ifconfig ix1
ix1: flags=1028943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC,LOWER_UP> metric 0 mtu 1500
    options=4813838<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,HWSTATS,MEXTPG>
    ether XX:XX:XX:XX:XX:XX
    media: Ethernet autoselect (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Code: 
# bhyve host, display netgraph configuration before vm startup
$ ngctl show ngbr0:
  Name: ngbr0           Type: bridge          ID: 00000171   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook    
  ----------      ---------       ---------    -------         ---------    
  link1           ix1             ether        00000095        upper        
  link0           ix1             ether        00000095        lower

Code: 
# bhyve host, display switch configuration
$ vm switch list
NAME   TYPE     IFACE                                ADDRESS  PRIVATE  MTU  VLAN  PORTS
ngbr0  netraph  netgraph,path=ngbr0:,peerhook=link2  n/a      n/a      n/a  n/a   n/a

bhyve vmComments
Code: 
network0_type="virtio-net"
network0_switch="ngbr0"
network0_mac="58:9c:fc:07:7c:be"
bhyve vm
network configuration

Code: 
# bhyve host, display bhyve vm attached to ngbr0:link2 after vm startup
$ ngctl show ngbr0:
  Name: ngbr0           Type: bridge          ID: 00000171   Num hooks: 3
  Local hook      Peer name       Peer type    Peer ID         Peer hook    
  ----------      ---------       ---------    -------         ---------    
  link2           <unnamed>       socket       00000180        vmlink       
  link1           ix1             ether        00000095        upper        
  link0           ix1             ether        00000095        lower

What am I missing here?

I am also aware of the Klara article, but I can't see where I'am wondering into the woods.
 
Just tried a different interface like ixl0, but that doesn't work either.

No one else using this feature ?
 
Looks like this is somehow related to the NIC driver (e.g. ix, ixl), I tried the same setup using em0 and it just works out-of-the-box. The bhyve guest is able to obtain an IP from the DHCPd at the router.

The following configure a working bridge on the default interface em0 and allows a bhyve guest to obtain an IP addr from the router.
Code: 
$ ngctl msg em0: setpromisc 1
$ ngctl msg em0: setautosrc 0
$ ngctl mkpeer em0: bridge lower link0
$ ngctl name em0:lower ngbr0
$ ngctl connect em0: ngbr0: upper link1
$ ifconfig em0 lro -tso4 -tso6 -vlanhwfilter -vlanhwtso
 
I wonder, if that got something to do with the fact that I have SRIOV and two VF configured for ix1 as well ?! 😅
 
You must log in or register to reply here.
Back
Top