Solved Allowing networking to bhyve guests through PF

stratacast1

Active Member

Reaction score: 22
Messages: 203

To start, I have looked this up and I have tried the suggestions here on the forums with no luck. I'm running FreeBSD 12.0 and using sysutils/vm-bhyve to do my vm management.

My main interface: em0
My switch: vm-public

My PF firewall rules:

Code:
ext_if="em0"

# Macros to define the set of TCP and UDP ports to open.
# Add additional ports or ranges separated by commas.
tcp_services = "{1022, 22}"
udp_services = "{1900, 5353, 32410, 32412, 32413, 32414, 25565}"

dhcp = "{ 67, 68 }"

# If you block all ICMP requests you will break things like path MTU
# discovery. These macros define allowed ICMP types. The additional
# ICMPv6 types are for neighbor discovery (RFC 4861)
icmp_types = "{echoreq, unreach}"

# Modulate the initial sequence number of TCP packets.
# Broken operating systems sometimes don't randomize this number,
# making it guessable.
tcp_state="flags S/SA keep state"
udp_state="keep state"

# Don't send rejections. Just drop.
set block-policy drop

# Exempt the loopback interface to prevent services utilizing the
# local loop from being blocked accidentally.
set skip on { lo0, vm-public }

# all incoming traffic on external interface is normalized and fragmented
# packets are reassembled.
scrub in on $ext_if all fragment reassemble

# Allow bhyve VMs
pass in quick on vm-public all

pass in  quick on vm-public proto udp from any port $dhcp to any port $dhcp
pass out quick on vm-public proto udp from any port $dhcp to any port $dhcp

###############################################

# set a default deny policy.
block in log all

# This is a desktop so be permissive in allowing outgoing connections.
pass out quick modulate state

# Enable antispoofing on the external interface
antispoof for $ext_if inet

# block packets that fail a reverse path check. we look up the routing
# table, check to make sure that the outbound is the same as the source
# it came in on. if not, it is probably source address spoofed.
block in from urpf-failed to any

# drop broadcast requests quietly.
block in quick on $ext_if from any to 255.255.255.255

# Allow the services defined in the macros at the top of the file
pass in on $ext_if inet proto tcp from any to any port $tcp_services $tcp_state

pass in on $ext_if inet proto udp from any to any port $udp_services $udp_state

# Allow ICMP
pass inet proto icmp all icmp-type $icmp_types keep state
To my understanding, the pass in quick on vm-public all should do the trick but alas, no luck. Thoughts?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 7,073
Messages: 29,218

You have a set skip on vm-public in your rules. So all your vm-public rules are ignored.

Code:
     set skip on <ifspec>
           List interfaces for which packets should not be filtered.  Packets
           passing in or out on such interfaces are passed as if pf was
           disabled, i.e. pf does not process them in any way.  This can be
           useful on loopback and other virtual interfaces, when packet
           filtering is not desired and can have unexpected effects.  For
           example:

                 set skip on lo0
See pf.conf(5)
 
OP
OP
stratacast1

stratacast1

Active Member

Reaction score: 22
Messages: 203

Oops that was a stupid blunder. However I've tried pass in and pass out quick on vm-public still and still no networking. Same trying to pass all on my tap interfaces that are created
 
OP
OP
stratacast1

stratacast1

Active Member

Reaction score: 22
Messages: 203

Okay, this is definitely a DHCP issue. I ran my machine with the firewall rules disabled, got my IP address, then enabled my firewall rules again and I was able to do all networking. So why is my firewall blocking only DHCP? Even when I have pass in/out quick all and specific rules allowing DHCP?

UPDATE: My router is also offering the address it wants. I'm seeing the request, ACK, and offer packets. So something is blocking it from actually accepting the IP on the firewall
 
OP
OP
stratacast1

stratacast1

Active Member

Reaction score: 22
Messages: 203

Found it. Simply allowing 67,68 on the bridge won't do. Looks like I also had to allow 68 on my main interface, where I only had 67 allowed. My firewalling skills are subpar at best these days. I should probably bust out that PF book I have 😅
 
Top