Hello everyone
So I have a dedicated server with Hetzner running FreeBSD 12.1-RELEASE-p3 currently not running PF firewall.
I have a /29 block of addresses assigned to me and I am trying to use them for Bhyve VM's but I am unsure how to go about it.
I have read the following form threads before posting:
I followed the instructions from the Hetzner wiki on FreeBSD addressing and can confirm that the address work when configured in /etc/rc.conf - That is they all route to the dedicated server as expected.
From my understanding of reading the above FreeBSD threads to use public IP addresses in Bhyve I have to remove the related ifconfig entries from /etc/rc.conf
So I commented out the ifconfig entries from /etc/rc.conf and configured Bhyve using the following guide.
I used the following commands in order.
The contents of /dn42/bhyve/lonniemason/lonniemason.conf are:
Output of
Output of
So I proceeded with the installation of Debian knowing that this is not going to work because without the related ifconfig entries in /etc/rc.conf the FreeBSD system is unaware of the /29 Subnet.
If FreeBSD is unaware of the /29 Subnet then how is Bhyve expected to be aware of the /29 Subnet and how to route it.
Set IP Address:
Set Netmask:
Set gateway:
Unable to reach gateway as expected:
The /29 block of addresses assigned to me by Hetzner uses my main dedicated server IP as the default gateway, even though the gateway address appears to be in a different subnet - not sure how that works.
The current contents of my
Output of
Output of
So at the very least I suspect that this is related to routing i.e somehow Bhyve needs to be aware of the /29 address space and how to route it.
Additionaly I am not 100% sure if my Bhyve switch config is correct for the scenario that I am trying to achieve.
Also I should note that I am new to FreeBSD but have had a good look at the FreeBSD handbook.
Any help would be greatly appreciated thanks
Lonnie Mason
So I have a dedicated server with Hetzner running FreeBSD 12.1-RELEASE-p3 currently not running PF firewall.
I have a /29 block of addresses assigned to me and I am trying to use them for Bhyve VM's but I am unsure how to go about it.
I have read the following form threads before posting:
IPv4 routing on Hetzner server
Hi all, I have a dedicated server at Hetzner running FreeBSD 11. Hetzner give you a single IPv4 address with their servers, but you can apply for more, so I also have a /28 on a different subnet, and Hetzner route that /28 through the original single IPv4 address. What I'm trying to do is use...
forums.freebsd.org
Bhyve, Public IP on guest visible to all. How?
Hello, I want guest VM to be fully accessible to the internet. Freebsd 12 with Bhyve, installed windows as guest. Set up static IP. Internet is working. Online tools report good IP (the one from guest, not host) Guest gateway is set to host IP. When i set gateway to the one i got from Hetzner...
forums.freebsd.org
How to assign public IP to a bhyve virtual instance?
Hi, I'm trying to figure out to assign a real IP address to a bhyve vm running Ubuntu. (Real IP address means a public facing IP address assigned by the provider.) I add: vm switch add public em0 Then, I should omit: ifconfig_igb0_alias0="17.28.44.219 netmask n.n.n.n" in the rc.conf...
forums.freebsd.org
I followed the instructions from the Hetzner wiki on FreeBSD addressing and can confirm that the address work when configured in /etc/rc.conf - That is they all route to the dedicated server as expected.
Hetzner Docs
Instructions, documentation and helpful information on all panels and products of Hetzner Online GmbH
wiki.hetzner.de
From my understanding of reading the above FreeBSD threads to use public IP addresses in Bhyve I have to remove the related ifconfig entries from /etc/rc.conf
Code:
ifconfig_em0_alias0="inet xx.xx.xx.xxx netmask xxx.xxx.xxx.xxx"
#additional entries omitted for brevity
So I commented out the ifconfig entries from /etc/rc.conf and configured Bhyve using the following guide.
GitHub - churchers/vm-bhyve: Shell based, minimal dependency bhyve manager
Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.
github.com
Code:
1. pkg install vm-bhyve grub2-bhyve
2. zfs create dn42/bhyve
3. sysrc vm_enable="YES"
4. sysrc vm_dir="zfs:/dn42/bhyve"
5. vm init
6. cp /usr/local/share/examples/vm-bhyve/* /dn42/bhyve/.templates/
7. vm switch create public
8. vm switch add public em0
9. vm iso https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.3.0-amd64-netinst.iso
10. vm create lonniemason
11. vm install lonniemason debian-10.3.0-amd64-netinst.iso
12. vm console lonniemason
The contents of /dn42/bhyve/lonniemason/lonniemason.conf are:
Code:
loader="grub"
cpu=1
memory=1024M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="lmsystem.img"
grub_run_partition="1"
grub_run_dir="/boot/grub"
uuid="45ff5cbd-8f24-11ea-81ab-60a44c22a490"
network0_mac="58:9c:fc:00:bd:0d"
Output of
vm switch list
Code:
NAME TYPE IFACE ADDRESS PRIVATE MTU VLAN PORTS
public standard vm-public - no - - em0
vm switch info public
Code:
------------------------
Virtual Switch: public
------------------------
type: standard
ident: vm-public
vlan: -
physical-ports: em0
bytes-in: 2009995 (1.916M)
bytes-out: 3222092 (3.072M)
virtual-port
device: tap0
vm: lonniemason
So I proceeded with the installation of Debian knowing that this is not going to work because without the related ifconfig entries in /etc/rc.conf the FreeBSD system is unaware of the /29 Subnet.
If FreeBSD is unaware of the /29 Subnet then how is Bhyve expected to be aware of the /29 Subnet and how to route it.
Set IP Address:
Set Netmask:
Set gateway:
Unable to reach gateway as expected:
The /29 block of addresses assigned to me by Hetzner uses my main dedicated server IP as the default gateway, even though the gateway address appears to be in a different subnet - not sure how that works.
The current contents of my
/etc/rc.conf
are
Code:
hostname="lonniemason"
keymap="uk.kbd"
ifconfig_em0="inet 46.4.50.x netmask 255.255.255.192"
defaultrouter="46.4.50.1"
ifconfig_em0_ipv6="inet6 accept_rtadv"
sshd_enable="YES"
powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
gateway_enable="YES"
vm_enable="YES"
vm_dir="zfs:dn42/bhyve"
#ifconfig entries are commented out
#ifconfig_em0_alias0="inet 49.12.49.xxx netmask 255.255.255.248" #Network ID cannot be used by a host as an IP address?
#ifconfig_em0_alias1="inet 49.12.49.xxx netmask 255.255.255.255" #Usable address
#ifconfig_em0_alias2="inet 49.12.49.xxx netmask 255.255.255.255" #Usable address
#ifconfig_em0_alias3="inet 49.12.49.xxx netmask 255.255.255.255" #Usable address
#ifconfig_em0_alias4="inet 49.12.49.xxx netmask 255.255.255.255" #Usable address
#ifconfig_em0_alias5="inet 49.12.49.xxx netmask 255.255.255.255" #Usable address
#ifconfig_em0_alias6="inet 49.12.49.xxx netmask 255.255.255.255" #Usable address
#ifconfig_em0_alias7="inet 49.12.49.xxx netmask 255.255.255.255" #Broadcast addressxxx
Output of
ifconfig
Code:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=812099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
ether 60:a4:4c:22:a4:90
inet 46.4.50.x netmask 0xffffffc0 broadcast 46.4.50.xx
inet6 fe80::62a4:4cff:fe22:a490%em0 prefixlen 64 scopeid 0x1
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 4a:68:0d:fa:ab:bb
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge vm-switch viid-4c918@
nd6 options=1<PERFORMNUD>
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vmnet-lonniemason-0-public
options=80000<LINKSTATE>
ether 00:bd:97:b4:02:00
groups: tap vm-port
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 1683
Output of
netstat -r
Code:
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default static.1.50.4.46.c UGS em0
46.4.50.0/26 link#1 U em0
dn42.lonniemason.n link#1 UHS lo0
localhost link#2 UH lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 localhost UGRS lo0
localhost link#2 UH lo0
::ffff:0.0.0.0/96 localhost UGRS lo0
fe80::/10 localhost UGRS lo0
fe80::%em0/64 link#1 U em0
fe80::62a4:4cff:fe link#1 UHS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 localhost UGRS lo0
So at the very least I suspect that this is related to routing i.e somehow Bhyve needs to be aware of the /29 address space and how to route it.
Additionaly I am not 100% sure if my Bhyve switch config is correct for the scenario that I am trying to achieve.
Also I should note that I am new to FreeBSD but have had a good look at the FreeBSD handbook.
Any help would be greatly appreciated thanks
Lonnie Mason