Public IP on bhyve guest

I'm trying to assign a public IP to a guest managed by bhyve.

My host gave me 2 public IPs to use, which I'll refer to as (to obfuscate the real IP):
  • 111.111.111.111 (host)
  • 222.222.222.222 (webserver vm)
The first, 111.111.111.111 is assigned to the host's interface, which has internet working as expected. The second IP should be assigned to the VM, which I have set up in the VM's /etc/rc.conf:

Code:
ifconfig_vtnet0="inet 222.222.222.222 netmask 255.255.255.0"

I have attached the VM to the public switch:

Code:
------------------------                  
Virtual Switch: public                    
------------------------                  
  type: auto                              
  ident: bridge0                          
  vlan: -                                  
  nat: yes                                
  physical-ports: em0                      
  bytes-in: 856834 (836.751K)              
  bytes-out: 20904094 (19.935M)            

  virtual-port                            
    device: tap1                          
    vm: database                          

  virtual-port                            
    device: tap0                          
    vm: webserver

My pf.conf on the host looks as follows:

Code:
include "/var/vm/.config/pf-nat.conf"

The VM can't ping to the outside, and can't be pinged from the outside either. This is my first time trying to virtualize with FreeBSD, so I have no idea how to troubleshoot this issue other than searching on the forums. I've read a couple forum posts but no solution there seems to fix my problem.

Any help would be greatly appreciated!
 
Turn off NAT on the public switch. There's no need for it.

vm switch nat public off

Note however that this will cause your database VM to be attached to the big bad internet too. You'll want to create two switches, one public; attached to the host's em0 and the web VM with NAT switched off. This will provide direct access to the internet for the webserver. Create a second switch, internal for example, and bind it to both the web and DB VMs. You can enable NAT on the internal switch if the DB VM needs to access the internet (for updates for example). The web server will need to have 2 interfaces, one attached to public and the other interface to internal. The web server will be able to talk to the DB server via the internal connection.
 
I added the second switch and the VMs can communicate with eachother over the second switch (which has NAT enabled).

Code:
NAME            TYPE       IDENT       VLAN      NAT          PORTS
public          auto       bridge0     -         -            em0
private         auto       bridge1     -         yes          -

The static IP is assigned in the VM on the interface that's connected to the public switch, but I still cant seem to reach it. The VM can ping to the outside, but if I remove the private interface (with the NAT) I can no longer ping the outside from the webserver VM. The error given is ping: sendto: No route to host.

If I try to ping the vm with the public IP from the host machine ( ping 222.222.222.222), I get a different error:
Code:
36 bytes from localhost (127.0.0.1): Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 3a45   0 0000  01  01 16d9 111.111.111.111  222.222.222.222
 
Generaly, host companies provides aditional IP addresses, that must be "attached" to an virtual MAC address. I don't know your host company, but at OVH.net (my host company) and several others, that's how it works...

My host cmd (pay attention at virtio-net,tap0,mac=00:50:56:04:2c:12)
Code:
bhyve -c 2 -m 8G -H -A -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0,mac=00:50:56:04:2c:12 -s 3:0,virtio-blk,/dev/zvol/zfs0/vm-cloud01 -l com1,/dev/nmdm0A cloud01

My guest IP config
Code:
ifconfig_vtnet0="inet PUBLIC_IP netmask 255.255.255.255"
static_routes="net1 net2"
route_net1="-net HOST_GATEWAY_IP/32 PUBLIC_IP"
route_net2="default HOST_GATEWAY_IP"

Good luck
 
I have Hetzner as host, and setting the virtual MAC address did the trick. Thanks a lot!
 
At home I have five public IPs: First to be used by my router WAN and the others are configured as virtual IPs on it. My VMs have static LAN IPs and I assign to the servers I want to expose to the internet one of the virtual Public IPs, via 1 to 1 NAT, and only forward the ports I need to be used. That way is (1) simple, as no configuration is required on the bhyve side and (2) secure as only the needed ports are open, and (3) centralized management.

Router/Firewall in use is pfSense, that I was running as a bhyve VM as well.
 
That is not the subject of this thread. The issue is already solved too.

Please read, before post...
 
Back
Top