Solved pf on one interface working but non of my virtual machines are able to see out

IF have the following configuration file:
Code:
## Set your public interface ##
ext_if="igb0"

## Set your server public IP address ##
ext_if_ip="192.168.81.1"

## Set and drop these IP ranges on public interface ##
martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \
       10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
       0.0.0.0/8, 240.0.0.0/4 }"

## Set http(80)/https (443) port here ##
webports = "{http, https,8443,119,561,110,143,993,995,21,1645,1646,1812,1813,23,
79,25,465,587,53,513,5900:5999}"

## enable these services ##
int_tcp_services = "{domain, ntp, smtp,nntp, smtps,submission, www, https, ftp,
ssh,110,143,636,993,995,443,561,7500,8443,10000,20000,4610,3389,5900:5999,23}"
int_udp_services = "{domain, ntp ,1645,1646,1812,1813 }"


## Skip loop back interface - Skip all PF processing on interface ##
set skip on lo

## Sets the interface for which PF should gather statistics such as bytes in/out
 and packets passed/blocked ##
set loginterface $ext_if

# Deal with attacks based on incorrect handling of packet fragments
scrub in all

## Set default policy ##
block return in log all
block out all


# Drop all Non-Routable Addresses
block drop in quick on $ext_if from $martians to any
block drop out quick on $ext_if from any to $martians

## Blocking spoofed packets
antispoof quick for $ext_if

pass in inet proto tcp to $ext_if port 22

# Allow Ping-Pong stuff. Be a good sysadmin
#pass inet proto icmp icmp-type echoreq

icmp_types = "{ echoreq, unreach }"
pass inet proto icmp all icmp-type $icmp_types keep state
# allow out the default range for traceroute(8):
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep stat
e

# All access to our Nginx/Apache/Lighttpd Webserver ports
pass proto tcp from any to $ext_if port $webports
pass proto udp from any to $ext_if port $webports

# Allow essential outgoing traffic
pass out quick on $ext_if proto tcp to any port $int_tcp_services
pass out quick on $ext_if proto udp to any port $int_udp_services

block quick from <bruteforce>
pass quick proto { tcp, udp } from any to any port ssh \
    flags S/SA keep state \
    (max-src-conn 15, max-src-conn-rate 5/3, \
    overload <bruteforce> flush global)
end configuration
just a block of bridges IPs are missing from 192.168.81.52 to 192.168.81.68 using bridge0 and tap0 to tap15 and cannot get outside.

What needs to be fixed?
 
I'm guessing you have igb0 -> bridge0 -> tap0-15? And the 192.168.81.1 address is assigned to igb0? Move that address to bridge0 instead.
 
you are saying let the bridge0 and igb0 have the same IP them. Did try that and the server went into a loop. Set bridge0 back to 192.168.81.60 .
 
another subquestion, has anyone set up proftpd inside such a pf firewall? answered, just that client ftp ware need to communicate with the server and not assume the ports they need
 
Last edited:
No, I said to move the IP address from igb0 to bridge0. Don't assign any address to igb0.


Tried that on a dummy machine 192.168.81.14 with taps of 192.168.81.15 to 192.168.81.18 and after the remote reboot cannot that server remotely. even a ping does not work from the lan. next suggestion? This is FreeBSD 12.0
 
Starting from square 1:

the pf.conf file
Code:
## Set your public interface ##
ext_if="igb0"
int_if="bridge0"
## Set your server public IP address ##
ext_if_ip="192.168.81.1"
int_if_ip="192.168.81.60"

proxy="127.0.0.1"
proxyport="8021"

sl="192.168.81.52"
fedora="192.168.81.53"
centos="192.168.81.54"
debian="192.168.81.55"
ubuntu="192.168.81.57"
win2019="192.168.81.58"
kali="192.168.81.59"
oracle="192.168.81.61"

vhosts =" { 192.168.81.52, 192.168.81.53, 192.168.81.54, \
           192.168.81.55 , 192.168.81.57, 192.168.81.58, \
           192.168.81.59,192.168.81.61 }"
## Set and drop these IP ranges on public interface ##
martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \
       10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
       0.0.0.0/8, 240.0.0.0/4 }"

## Set http(80)/https (443) port here ##
webports = "{http, https,8443,119,561,110,143,993,995,21,1645,1646,1812,1813,23,79,25,465,587,53,513,783,3310,4610,3389,69,10000,20000,43,63,4321,8000:8100,5900:5999,49150:61000}"

## enable these services ##
int_tcp_services = "{domain, ntp, smtp,nntp, smtps,submission, www, https, ftp, ssh,110,143,636,993,995,443,561,783,3310,7500,8443,10000,20000,43,63,4321,4610,3389,8000:8100,5900:5999,23,49150:61000}"
int_udp_services = "{domain, ntp,69,1645,1646,1812,1813 }"

 
## Skip loop back interface - Skip all PF processing on interface ##
set skip on lo
set skip on bridge0
set skip on tap0
set skip on tap1
set skip on tap2
set skip on tap3
set skip on tap4
set skip on tap5
set skip on tap6
set skip on tap7
set skip on tap8
set skip on tap9
set skip on tap10
set skip on tap11
set skip on tap12
set skip on tap13
set skip on tap14
set skip on tap15

 
## Sets the interface for which PF should gather statistics such as bytes in/out and packets passed/blocked ##
set loginterface $ext_if
set fingerprints "/etc/pf.os" 

# Deal with attacks based on incorrect handling of packet fragments 
scrub in all

###################  TRASLATION #############

#### NAT and RDR start
nat on $ext_if from $int_if to any -> ($ext_if)
nat on $ext_if from $sl to any -> ($ext_if)
nat on $ext_if from $fedora to any -> ($ext_if)
nat on $ext_if from $centos to any -> ($ext_if)
nat on $ext_if from $debian to any -> ($ext_if)
nat on $ext_if from $ubuntu to any -> ($ext_if)
nat on $ext_if from $win2019 to any -> ($ext_if)
nat on $ext_if from $kali to any -> ($ext_if)
nat on $ext_if from $oracle to any -> ($ext_if)

nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
 
# Redirect ftp traffic to proxy
rdr pass proto tcp from any to any port ftp -> $proxy port $proxyport


## Set default policy ##
block return in log all
block out all

# We need to have an anchor for ftp-proxy
anchor "ftp-proxy/*"
pass out proto tcp from $proxy to any port 21
pass out on $ext_if inet proto {tcp, udp} from $ext_if to any port ftp:ftp-proxy
pass in on egress proto tcp to port 21
pass in on egress proto tcp to port > 49151
pass out quick on egress inet proto tcp from any to 192.168.81.1 flags S/SA
pass out quick on egress inet proto tcp from any to 192.168.81.3 flags S/SA

pass in quick on bridge0 all
pass quick on tap0 all 
pass quick on tap1 all 
pass quick on tap2 all 
pass quick on tap3 all 
pass quick on tap4 all 
pass quick on tap5 all 
pass quick on tap6 all 
pass quick on tap7 all 
pass quick on tap8 all 
pass quick on tap9 all 
pass quick on tap10 all 
pass quick on tap11 all 
pass quick on tap12 all 
pass quick on tap13 all 
pass quick on tap14 all 
pass quick on tap15 all 

# Drop all Non-Routable Addresses 
block drop in quick on $ext_if from $martians to any
block drop out quick on $ext_if from any to $martians
block drop in quick on $vhosts from $martians to any
block drop out quick on $vhosts from any to $martians
 
## Blocking spoofed packets
antispoof quick for $ext_if
antispoof quick for $vhosts
 
# Open SSH port which is listening on port 22 from VPN 139.xx.yy.zz Ip only
# I do not allow or accept ssh traffic from ALL for security reasons 
#pass in quick on $ext_if inet proto tcp from 192.168.81.0/24 to $ext_if_ip port = ssh flags S/SA keep state label "USER_RULE: Allow SSH from 139.xxx.yyy.zzz"
## Use the following rule to enable ssh for ALL users from any IP address #
## pass in inet proto tcp to $ext_if port ssh
### [ OR ] ###
pass in inet proto tcp to $ext_if port 22 
pass in inet proto tcp to $vhosts port 22 


# Allow Ping-Pong stuff. Be a good sysadmin 
#pass inet proto icmp icmp-type echoreq

icmp_types = "{ echoreq, unreach }"
pass inet proto icmp all icmp-type $icmp_types keep state
# allow out the default range for traceroute(8):
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state
pass out on $int_if inet proto udp from any to any port 33433 >< 33626 keep state
pass out on $vhosts inet proto udp from any to any port 33433 >< 33626 keep state
  
# All access to our Nginx/Apache/Lighttpd Webserver ports 
pass proto tcp from any to $ext_if port $webports
pass proto udp from any to $ext_if port $webports
pass proto tcp from any to $vhosts port $webports
pass proto udp from any to $vhosts port $webports

 
# Allow essential outgoing traffic 
pass out quick on $ext_if proto tcp to any port $int_tcp_services
pass out quick on $ext_if proto udp to any port $int_udp_services
pass out quick on $vhosts proto tcp to any port $int_tcp_services
pass out quick on $vhosts proto udp to any port $int_udp_services
 
# Add custom rules below
block quick from <bruteforce>
pass quick proto { tcp, udp } from any to any port ssh \
    flags S/SA keep state \
    (max-src-conn 15, max-src-conn-rate 5/3, \
    overload <bruteforce> flush global)

The rc.conf file
Code:
ifconfig_igb0="inet 192.168.81.1 netmask 255.255.255.0"
defaultrouter="192.168.81.2"
zfs_enable="YES"
zfsd_enable="YES"
sshd_enable="YES"
sshguard_enable="YES"
moused_enable="YES"
ntpd_enable="YES"
powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
slapd_enable="YES"
slapd_flags='-h "ldapi://%2fvar%2frun%2fopenldap%2fldapi/ ldap://0.0.0.0/"'
slapd_sockets="/var/run/openldap/ldapi"
radiusd_enable="YES"
clamav_clamd_enable="YES"
clamd_enable="YES"
postgresql_enable="YES"
webmin_enable="YES"
usermin_enable="YES"
mysql_enable="YES"
#sysadm_start="YES"
#sysadm_rest_Enable="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msq_queue_enable="NO"
saslauthd_enable="YES"
exim_enable="YES"
dovecot_enable="YES"
mailscannner="YES"
mailscanner_user="exim"
proftpd_enable="YES"
ftpproxy_enable="YES"
ftpproxy_flags=" -D7 -v"
spamd_enable="YES"
spamd_flags="-u spamd -H /var/spool/spamd -d -D --max-children=5 --max-conn-per-child=5 --user-config --syslog=/var/log/spamd.log --pidfile=/var/run/spamd.pid --listen-ip=192.168.81.1 --port=783 -A 192.168.81.1,192.168.81.3,127.0.0.1 &"
apache24_enable="yes"
devfs_system_ruleset="system"
mongod_enable="YES"
mongod_config="/usr/local/etc/mongodb.conf"
mongod_dbpath="/var/db/mongodb"
redis_enable="YES"
linux_enable="YES"
cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5 tap6 tap7 tap8 tap9 tap10 tap11 tap12 tap13 tap14 tap15"  
ifconfig_bridge0="addm igb0 addm tap0 addm tap1 addm tap2 addm tap3 addm tap4 addm tap5 addm tap6 addm tap7 addm tap8 addm tap9 addm tap10 addm tap11 addm tap12 addm tap13 addm tap14 addm tap15 up"
vm_enable="YES"
vm_dir="/usr/vm/"
pf_enable="YES"
pf_rules="/etc/pf.conf"
pflog_enable="YES"
pflog_logfile="/var/log/pflog"
#vboxdrv_load="YES"
#vboxnet_enable="YES" 
xrdp_enable="YES"
xrdp_sesman_enable="YES"
firebird_enable="YES"
firebird_mode="superserver"    
nfs_client_enable="YES"
named_enable="YES"
inetd_enable="YES"
dbus_enable="YES"
hald_enable="YES"
bitcoind_enable="YES"
syslog_ng_enable="YES"
syslog_enable="NO"
c_icap_enable="YES"
xencommons_enable="yes"
autobridge_interfaces="bridge0"
autobridge_bridge0="igb0"
puppet_enable="YES"
puppetdb_enable="YES"
kqemu_enable="YES" 
lookup_domain_enable="YES"
atop_enable="YES"
rpcbind_enable="YES"
suricata_enable="YES"
suricata_interface="igb0"
suricata_divertport="8000"
suricata_netmap="YES"
openvpn_enable="YES"
ntopng_enable="YES"
tor_enable="NO"
microcode_update_enable="YES"
ntpd_program="/usr/local/sbin/ntpd"
ntpdate_program="/usr/local/sbin/ntpdate"
bsdstats_enable="YES"
smartd_enable="YES"

egrep of rc.conf file


The rc.local file
Code:
/usr/gnurad/sbin/radiusd -A -i 192.168.81.1 -p 1645 ; 
/usr/dovecot2/sbin/dovecot
/usr/sbin/inetd -wW -R 65536

#/usr/local/bin/netserver

/sbin/ifconfig igb0 alias 10.0.0.2 netmask 0xffffffff
/sbin/ifconfig igb0 alias 10.0.0.3 netmask 0xffffffff
/sbin/ifconfig igb0 alias 10.0.0.4  netmask 0xffffffff
/sbin/ifconfig igb0 alias 10.0.0.5 netmask 0xffffffff
/sbin/ifconfig igb0 alias 10.0.0.6 netmask 0xffffffff
/sbin/ifconfig igb0 alias 10.0.0.7 netmask 0xffffffff
/sbin/ifconfig igb0 alias 10.0.0.8 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.50 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.51 netmask 0xffffffff
/sbin/ifconfig tap8 inet 192.168.81.52 netmask 0xffffff00
/sbin/ifconfig tap0 inet 192.168.81.53 netmask 0xffffff00
/sbin/ifconfig tap1 inet 192.168.81.54 netmask 0xffffff00
/sbin/ifconfig tap7 inet 192.168.81.55 netmask 0xffffff00
/sbin/ifconfig tap2 inet 192.168.81.56 netmask 0xffffff00
/sbin/ifconfig tap4 inet 192.168.81.57 netmask 0xffffff00
/sbin/ifconfig tap3 inet 192.168.81.58 netmask 0xffffff00
/sbin/ifconfig tap6 inet 192.168.81.59 netmask 0xffffff00
/sbin/ifconfig bridge0 alias 192.168.81.60 netmask 0xffffff00
/sbin/ifconfig tap5 inet 192.168.81.61 netmask 0xffffff00
/sbin/ifconfig tap9 inet 192.168.81.62 netmask 0xffffff00
/sbin/ifconfig tap10 inet 192.168.81.63 netmask 0xffffff00
/sbin/ifconfig tap11 inet 192.168.81.64 netmask 0xffffff00
/sbin/ifconfig tap12 inet 192.168.81.65 netmask 0xffffff00
/sbin/ifconfig tap13 inet 192.168.81.66 netmask 0xffffff00
/sbin/ifconfig tap14 inet 192.168.81.67 netmask 0xffffff00
/sbin/ifconfig tap15 inet 192.168.81.68 netmask 0xffffff00
/sbin/ifconfig igb0 alias 192.168.81.69 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.70 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.71 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.72 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.73 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.74 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.75 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.76 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.77 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.78 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.79 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.80 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.81 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.82 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.83 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.84 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.85 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.86 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.87 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.88 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.89 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.90 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.91 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.92 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.93 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.94 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.95 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.96 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.97 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.98 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.99 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.100 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.101 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.102 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.103 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.104 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.105 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.106 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.107 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.108 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.109 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.110 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.111 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.112 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.113 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.114 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.115 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.116 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.117 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.118 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.119 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.120 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.121 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.122 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.123 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.124 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.125 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.126 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.127 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.128 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.129 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.130 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.131 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.132 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.133 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.134 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.135 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.136 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.137 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.138 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.139 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.140 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.141 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.142 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.143 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.144 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.145 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.146 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.147 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.148 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.149 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.150 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.151 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.152 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.153 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.154 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.155 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.156 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.157 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.158 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.159 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.160 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.161 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.162 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.163 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.164 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.165 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.166 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.167 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.168 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.169 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.170 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.171 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.172 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.173 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.174 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.175 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.176 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.177 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.178 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.179 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.180 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.181 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.182 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.183 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.184 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.185 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.186 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.187 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.188 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.189 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.190 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.191 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.192 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.193 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.194 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.195 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.196 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.197 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.198 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.199 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.200 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.201 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.202 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.203 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.204 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.205 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.206 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.207 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.208 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.209 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.210 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.211 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.212 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.213 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.214 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.215 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.216 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.217 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.218 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.219 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.220 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.221 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.222 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.223 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.224 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.225 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.226 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.227 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.228 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.229 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.230 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.231 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.232 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.233 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.234 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.235 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.236 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.237 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.238 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.239 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.240 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.241 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.242 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.243 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.244 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.245 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.246 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.247 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.248 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.249 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.250 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.251 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.252 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.253 netmask 0xffffffff
/sbin/ifconfig igb0 alias 192.168.81.254 netmask 0xffffffff

/sbin/sysctl compat.linux.osrelease=4.18.16
/sbin/sysctl net.local.stream.recvspace=16384
/sbin/sysctl net.link.tap.up_on_open=1

#cd /usr/home/doctor/vm/

/usr/sbin/bhyve -ASHP -s 0:0,hostbridge -s 31:0,lpc \
-s 2:0,virtio-net,tap8 -s 3:0,virtio-blk,/usr/vm/images/scientific \
 -c 4 -m 512M \
-s 29,fbuf,tcp=0.0.0.0:5952,w=1024,h=768,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
scientificguest &


/usr/sbin/bhyve -ASHP  \
 -s 0:0,hostbridge \
 -s 31:0,lpc \
 -s 2:0,virtio-net,tap0 \
 -s 3:0,virtio-blk,/usr/vm/images/fedora \
 -c 4 -m 1024M \
 -s 29,fbuf,tcp=0.0.0.0:5953,w=1024,h=768,wait \
 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
fedoraguest &


/usr/sbin/bhyve -ASHP  \
 -s 0:0,hostbridge \
 -s 31:0,lpc \
 -s 2:0,virtio-net,tap1 \
 -s 3:0,virtio-blk,/usr/vm/images/centos \
 -c 4 -m 512M \
 -s 29,fbuf,tcp=0.0.0.0:5954,w=1024,h=768,wait \
 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
 centosguest &


/usr/sbin/bhyve -ASHP  \
 -s 0:0,hostbridge \
 -s 31:0,lpc \
 -s 2:0,virtio-net,tap7  \
 -s 3:0,virtio-blk,/usr/vm/images/debian \
 -s 4:0,ahci-cd,./.iso/super_grub2_disk_hybrid_2.02s10.iso \
 -c 4 -m 512M \
 -s 29,fbuf,tcp=0.0.0.0:5955,w=1024,h=768,wait \
 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debianguest &


/usr/sbin/bhyve -ASHP -w -u \
 -s 0:0,hostbridge \
 -s 31:0,lpc \
 -s 2:0,virtio-net,tap4 \
 -s 3:0,virtio-blk,/usr/vm/images/ubuntu \
 -c 4 -m 1024M \
 -s 29,fbuf,tcp=0.0.0.0:5957,w=1024,h=768,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
ubuntuguest &


bhyve -c 4  \
 -s 0,hostbridge \
 -s 3,nvme,/usr/vm/images/windows2019st \
 -s 10,virtio-net,tap3 \
 -s 20,virtio-rnd \
 -s 31,lpc \
 -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A \
  -s 29,fbuf,tcp=0.0.0.0:5958,w=1024,h=768,wait \
 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
 -m 1G -H -w  \
 windows2019st &


/usr/sbin/bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc \
-s 2:0,virtio-net,tap6 -s 3:0,virtio-blk,/usr/vm/images/kali \
 -s 4:0,ahci-cd,./.iso/super_grub2_disk_hybrid_2.02s10.iso \
 -c 4 -m 1024M \
-s 29,fbuf,tcp=0.0.0.0:5959,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
kaliguest &


/usr/sbin/bhyve -ASHP -s 0:0,hostbridge -s 31:0,lpc -s 2:0,virtio-net,tap5 -s 3:0,virtio-blk,/usr/vm/images/oracle -s 29,fbuf,tcp=0.0.0.0:5961,w=1024,h=768,wait -c 4 -m 512M -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd oracleguest &


cd /root



most everything is working save the VMs seeing the outside world. Here is the oddity:

I can ssh and rcp into the VMs but a ping and a traceroute and attempts to update check.

What is pf.conf restricting the VMs?
 
And here how to solve the issue:

This does act as server with mutliple functions so the pf.conf file for server one is

-----------------------------------------------------------------------------------------

## Set your public interface ##
ext_if="igb0"
##Internal bridge for virtually hosted machines
int_if="bridge0"
## Set your server public IP address ##
ext_if_ip="192.168.81.1"
int_if_ip="192.168.81.60"
#Proxy for FTP
proxy="127.0.0.1"
proxyport="8021"
#All virtal machines go here!
sl="192.168.81.52"
fedora="192.168.81.53"
centos="192.168.81.54"
debian="192.168.81.55"
ubuntu="192.168.81.57"
win2019="192.168.81.58"
kali="192.168.81.59"
oracle="192.168.81.61"
#In case you need a whole group
vhosts =" { 192.168.81.52, 192.168.81.53, 192.168.81.54, \
192.168.81.55 , 192.168.81.57, 192.168.81.58, \
192.168.81.59,192.168.81.61 }"
## Set and drop these IP ranges on public interface and any other troublemakers ##

martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \
10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
0.0.0.0/8, 240.0.0.0/4 }"

## Set http(80)/https (443) port here and other ports that need accessing ##
webports = "{http, https,8443,119,561,110,143,993,995,20,21,23,79,25,465,587,53,513,783,3310,4610,3389,69,10000,20000,43,63,4321,8000:8100,5900:5999,49150:61000}"
# Radius
radiusports = "{1645,1646,1812,1813 }"

## enable these services ##
int_tcp_services = "{domain, ntp, smtp,nntp, smtps,submission, www, https,20, ftp, ssh,110,143,636,993,995,443,561,783,3310,7500,8443,10000,20000,43,63,4321,4610,3389,8000:8100,5900:5999,23,49150:61000}"
int_udp_services = "{domain, ntp,69}"
int_radius_services = "{1645,1646,1812,1813 }"


## Skip loop back interface - Skip all PF processing on interface bridge and virtual hosts ##
set skip on lo
set skip on bridge0
set skip on tap0
set skip on tap1
set skip on tap2
set skip on tap3
set skip on tap4
set skip on tap5
set skip on tap6
set skip on tap7
set skip on tap8
set skip on tap9
set skip on tap10
set skip on tap11
set skip on tap12
set skip on tap13
set skip on tap14
set skip on tap15


## Sets the interface for which PF should gather statistics such as bytes in/out and packets passed/blocked ##
set loginterface $ext_if
set fingerprints "/etc/pf.os"

# Deal with attacks based on incorrect handling of packet fragments
scrub in all

################### TRANSLATION #############

#### NAT and RDR start
nat on $ext_if from $int_if to any -> ($ext_if)
nat on $int_if from $sl to any -> ($int_if)
nat on $int_if from $fedora to any -> ($int_if)
nat on $int_if from $centos to any -> ($int_if)
nat on $int_if from $debian to any -> ($int_if)
nat on $int_if from $ubuntu to any -> ($int_if)
nat on $int_if from $win2019 to any -> ($int_if)
nat on $int_if from $kali to any -> ($int_if)
nat on $int_if from $oracle to any -> ($int_if)

## PLease note for virtual machines you are passing the packects via the
## Virtual switch so treat as michine (tap) into switch (Bridge) into
## your macine acting as the host (exit)

nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"

# Redirect ftp traffic to proxy
rdr pass proto tcp from any to any port ftp -> $proxy port $proxyport


## Set default policy ##
block return in log all
block out all

# We need to have an anchor for ftp-proxy
anchor "ftp-proxy/*"
pass out proto tcp from $proxy to any port 20
pass out proto tcp from $proxy to any port 21
pass out on $ext_if inet proto {tcp, udp} from $ext_if to any port ftp:ftp-proxy
pass in on egress proto tcp to port 21
pass in on egress proto tcp to port 20
pass in on egress proto tcp to port > 49151
pass out quick on egress inet proto tcp from any to 192.168.81.1 flags S/SA
pass out quick on egress inet proto tcp from any to 192.168.81.3 flags S/SA

#set up virtual switch

pass in quick on bridge0 all
pass quick on tap0 all
pass quick on tap1 all
pass quick on tap2 all
pass quick on tap3 all
pass quick on tap4 all
pass quick on tap5 all
pass quick on tap6 all
pass quick on tap7 all
pass quick on tap8 all
pass quick on tap9 all
pass quick on tap10 all
pass quick on tap11 all
pass quick on tap12 all
pass quick on tap13 all
pass quick on tap14 all
pass quick on tap15 all

# Drop all Non-Routable Addresses
block drop in quick on $ext_if from $martians to any
block drop out quick on $ext_if from any to $martians
block drop in quick on $vhosts from $martians to any
block drop out quick on $vhosts from any to $martians

## Blocking spoofed packets
antispoof quick for $ext_if
antispoof quick for $vhosts

# Open SSH port which is listening on port 22 from VPN 139.xx.yy.zz Ip only
# I do not allow or accept ssh traffic from ALL for security reasons
#pass in quick on $ext_if inet proto tcp from 192.168.81.0/24 to $ext_if_ip port = ssh flags S/SA keep state label "USER_RULE: Allow SSH from 139.xxx.yyy.zzz"
## Use the following rule to enable ssh for ALL users from any IP address #
## pass in inet proto tcp to $ext_if port ssh
### [ OR ] ###
pass in inet proto tcp to $ext_if port 22
pass in inet proto tcp to $vhosts port 22


# Allow Ping-Pong stuff. Be a good sysadmin
icmp_types = "{ echoreq, unreach }"
pass inet proto icmp all icmp-type $icmp_types keep state
# allow out the default range for traceroute(8):
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state
pass out on $int_if inet proto udp from any to any port 33433 >< 33626 keep state
pass out on $vhosts inet proto udp from any to any port 33433 >< 33626 keep state

# All access to our Nginx/Apache/Lighttpd Webserver and other ports
pass proto tcp from any to $ext_if port $webports
pass proto udp from any to $ext_if port $webports
pass proto udp from any to $ext_if port $radiusports
pass proto tcp from any to $vhosts port $webports
pass proto udp from any to $vhosts port $webports


# Allow essential outgoing traffic
pass out quick on $ext_if proto tcp to any port $int_tcp_services
pass out quick on $ext_if proto udp to any port $int_udp_services
pass out quick on $ext_if proto udp to any port $int_radius_services
pass out quick on $vhosts proto tcp to any port $int_tcp_services
pass out quick on $vhosts proto udp to any port $int_udp_services

#For radius make certain for older syatems port 1645 and current 1812
pass in log quick on $ext_if proto tcp from any to any port = 1645 flags S/SA keep state
pass in log quick on $ext_if proto udp from any to any port = 1645 keep state
pass in log quick on $ext_if proto tcp from any to any port = 1812 flags S/SA keep state
pass in log quick on $ext_if proto udp from any to any port = 1812 keep state
pass out quick all flags S/SA keep state

# Add custom rules below
block quick from <bruteforce>
pass quick proto { tcp, udp } from any to any port ssh \
flags S/SA keep state \
(max-src-conn 15, max-src-conn-rate 5/3, \
overload <bruteforce> flush global)
## I wonder if sshguard works with pf.


---------------------------------------------------------------------------------------------------

And the secondary server pf.conf file is :

---------------------------------------------------------------------------------------------------------
## Set your public interface ##
ext_if="igb0"

## Set your server public IP address ##
ext_if_ip="192.168.81.3"

## Set and drop these IP ranges on public interface ##
martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \
10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
0.0.0.0/8, 240.0.0.0/4,43.241.16.0/22 }"

## Set http(80)/https (443) port here ##
webports = "{http, https,69,119,561,7500,443,10000,20000,20,21,23,79,25,465,587,53,513,783,3310,43,63,4321,8000:8100,5900:5999,49150:65530}"
webports = "{1645,1646,1812,1813}"

## enable these services ##
int_tcp_services = "{domain, ntp, smtp,nntp, smtps,submission, www, https, ftp,20, ssh,110,143,636,993,995,443,561,7500,8443,10000,20000,4610,3389,43,63,4321,5900:5999,23,49150:65530}"
int_udp_services = "{domain, ntp,69 }"
int_radius_services = "{1645,1646,1812,1813 }"


## Skip loop back interface - Skip all PF processing on interface ##
set skip on lo

## Sets the interface for which PF should gather statistics such as bytes in/out and packets passed/blocked ##
set loginterface $ext_if

# Deal with attacks based on incorrect handling of packet fragments
scrub in all

## Set default policy ##
block return in log all
block out all


# Drop all Non-Routable Addresses
block drop in quick on $ext_if from $martians to any
block drop out quick on $ext_if from any to $martians

## Blocking spoofed packets
antispoof quick for $ext_if

# Open SSH port which is listening on port 22 from VPN 139.xx.yy.zz Ip only
# I do not allow or accept ssh traffic from ALL for security reasons
#pass in quick on $ext_if inet proto tcp from 192.168.81.0/24 to $ext_if_ip port = ssh flags S/SA keep state label "USER_RULE: Allow SSH from 139.xxx.yyy.zzz"
## Use the following rule to enable ssh for ALL users from any IP address #
## pass in inet proto tcp to $ext_if port ssh
### [ OR ] ###
pass in inet proto tcp to $ext_if port 22

# Allow Ping-Pong stuff. Be a good sysadmin
#pass inet proto icmp icmp-type echoreq

icmp_types = "{ echoreq, unreach }"
pass inet proto icmp all icmp-type $icmp_types keep state
# allow out the default range for traceroute(8):
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state

# All access to our Nginx/Apache/Lighttpd Webserver ports
pass proto tcp from any to $ext_if port $webports
pass proto udp from any to $ext_if port $webports

# Allow essential outgoing traffic
pass out quick on $ext_if proto tcp to any port $int_tcp_services
pass out quick on $ext_if proto udp to any port $int_udp_services
pass out quick on $ext_if proto udp to any port $int_radius_services

#For radius make certain for older syatems port 1645 and current 1812
pass in log quick on $ext_if proto tcp from any to any port = 1645 flags S/SA keep state
pass in log quick on $ext_if proto udp from any to any port = 1645 keep state
pass in log quick on $ext_if proto tcp from any to any port = 1812 flags S/SA keep state
pass in log quick on $ext_if proto udp from any to any port = 1812 keep state
pass out quick all flags S/SA keep state
# Add custom rules below
block quick from <bruteforce>
pass quick proto { tcp, udp } from any to any port ssh \
flags S/SA keep state \
(max-src-conn 15, max-src-conn-rate 5/3, \
overload <bruteforce> flush global)



----------------------------------------------------------------------------------------------------------

Not for a broder system using PF!
 
Back
Top