Every now and then, somebody sends a large e-mail through my mail server and causes other connections to be very lagged until it clears. To prevent this, I've been working on implementing traffic shaping on a mail server running FreeBSD 7.1. To that end, I've built a custom kernel with ALTQ and CBQ enabled. That kernel has been running for several days with no trouble, so I'm ready to activate some traffic shaping rules. Since this is a live box and the only FreeBSD system I have available, I'm concerned about making an error that will cause pf to behave in an unexpected manner and cause problems for people trying to use the system. (If I activate the new pf rules and nobody can get mail, it'll be pretty obvious where the problem is... 
I'm more worried about breaking something silently that isn't noticed for days and results in adverse behavior.)
Could somebody who's more knowledgeable about pf and altq take a peek at the following and tell me if/how I screwed up.
My network configuration:
nfe0 is connected to a DSL modem.
tun0 is the virtual interface created by pppd.
vr0 is connected directly to an administration laptop. (telnetd and webmin only accept connections from this laptop.) Uses NAT.
rl0 is cabled to a wireless router so that our customers may use the line as a wifi hotspot. I only allow ports 80 and 443 to be used for surfing the web. Uses NAT.
Here is my current pf.conf; I built it from looking at examples of rule sets I found on the Internet. I don't understand about 5-10% of it, but it's been working well:
I built a list of all traffic that leaves the machine and decided upon somewhat arbitrary numbers for the bandwidth that should be reserved for each, making sure the total equals the 512k upstream speed on the DSL line:
DNS: 20k
Serving web pages: 40k
TCP ACKs: 40k
POP/IMAP: 100k
SMTP: 100k
Admin Laptop: 90k
WiFi Network: 50k
Everything else: 72k
Basically, I'd like each service listed above to have the corresponding amount of bandwidth available for it to use at any given time. If the upstream link is not saturated, any service can use the extra bandwidth.
By using a guide to traffic shaping with pf, I put traffic shaping rules into my pf.conf:
If anybody has comments or advice on the traffic shaping, I'd appreciate a reply. Likewise, if anybody has any suggestions for where I can improve the firewall, I'd appreciate those as well.
I'm more worried about breaking something silently that isn't noticed for days and results in adverse behavior.)Could somebody who's more knowledgeable about pf and altq take a peek at the following and tell me if/how I screwed up.
My network configuration:
nfe0 is connected to a DSL modem.
tun0 is the virtual interface created by pppd.
vr0 is connected directly to an administration laptop. (telnetd and webmin only accept connections from this laptop.) Uses NAT.
rl0 is cabled to a wireless router so that our customers may use the line as a wifi hotspot. I only allow ports 80 and 443 to be used for surfing the web. Uses NAT.
Here is my current pf.conf; I built it from looking at examples of rule sets I found on the Internet. I don't understand about 5-10% of it, but it's been working well:
Code:
# Set variables.
#ext_if = "nfe0"
ext_if = "tun0"
lcl_if = "lo0"
lcl_ip = "127.0.0.1/32"
int_if = "rl0"
int_ip = "192.168.20.1/24"
int_sip = "192.168.20.1/32"
adm_if = "vr0"
adm_ip = "192.168.5.2/32"
adm_sip = "192.168.5.1/32"
pop3_ports = "{ 110, 995 }"
imap_ports = "{ 143, 993 }"
smtp_ports = "{ 25, 2225 }"
bind_ports = "{ 53 }"
webi_ports = "{ 80 }"
webo_ports = "{ 443 }"
admn_ports = "{ 23, 10101 }"
icmp_types = "echoreq"
# Set defaults.
set block-policy return
#set loginterface $ext_if
set skip on $lcl_if
scrub in on { $ext_if, $int_if }
# Allow internal interfaces to get to the internet.
nat on $ext_if from $int_ip to any -> ($ext_if)
nat on $ext_if from $adm_ip to any -> ($ext_if)
#nat-anchor "ftp-proxy/*"
# Allow local nets to FTP out. Needed??? Doesn't seem to make any difference...
#rdr-anchor "ftp-proxy/*"
#rdr on $int_if proto tcp from any to any port 21 -> 127.0.0.1 port 8021
#rdr on $adm_if proto tcp from any to any port 21 -> 127.0.0.1 port 8021
# Block everything unless later explicitly allowed.
block in
# Keep state for established connections.
pass out keep state
# Allow local nets to FTP out. Needed??? Doesn't seem to make any difference...
#anchor "ftp-proxy/*"
# Protect against IP spoofing on local network segments.
antispoof quick for { $lcl_if $int_if $adm_if }
# Block inbound traffic from IPs not valid for each interface.
block in quick on ! $int_if inet from $int_ip to any
block in quick on ! $adm_if inet from $adm_ip to any
# Nobody else is me - block attempts to make us think so.
block in quick on $int_if inet from $int_sip to any
block in quick on $adm_if inet from $adm_sip to any
# Assume administrator knows what he is doing. Not necessarily true...
pass in quick on $adm_if from $adm_ip
pass in on $ext_if inet proto tcp from any to ($ext_if) port $pop3_ports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to ($ext_if) port $imap_ports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to ($ext_if) port $smtp_ports flags S/SA keep state
pass in on $ext_if inet proto { tcp, udp } from any to ($ext_if) port $bind_ports
pass in on $ext_if inet proto tcp from any to ($ext_if) port $webi_ports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to ($ext_if) port $webo_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $pop3_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $imap_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $smtp_ports flags S/SA keep state
pass in on $int_if inet proto { tcp, udp } from any to any port $bind_ports
pass in on $int_if inet proto tcp from any to any port $webi_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $webo_ports flags S/SA keep state
pass in inet proto icmp all icmp-type $icmp_types keep stateI built a list of all traffic that leaves the machine and decided upon somewhat arbitrary numbers for the bandwidth that should be reserved for each, making sure the total equals the 512k upstream speed on the DSL line:
DNS: 20k
Serving web pages: 40k
TCP ACKs: 40k
POP/IMAP: 100k
SMTP: 100k
Admin Laptop: 90k
WiFi Network: 50k
Everything else: 72k
Basically, I'd like each service listed above to have the corresponding amount of bandwidth available for it to use at any given time. If the upstream link is not saturated, any service can use the extra bandwidth.
By using a guide to traffic shaping with pf, I put traffic shaping rules into my pf.conf:
Code:
# Set variables.
#ext_if = "nfe0"
ext_if = "tun0"
lcl_if = "lo0"
lcl_ip = "127.0.0.1/32"
int_if = "rl0"
int_ip = "192.168.20.1/24"
int_sip = "192.168.20.1/32"
adm_if = "vr0"
adm_ip = "192.168.5.2/32"
adm_sip = "192.168.5.1/32"
pop3_ports = "{ 110, 995 }"
imap_ports = "{ 143, 993 }"
mail_ports = "{ 110, 995, 143, 993 }"
smtp_ports = "{ 25, 2225 }"
bind_ports = "{ 53 }"
webi_ports = "{ 80 }"
webo_ports = "{ 443 }"
admn_ports = "{ 23, 10101 }"
icmp_types = "echoreq"
# Activate alternate queuing
altq on tun0 cbq bandwidth 512Kb queue { standard_out, dns_out, http_out, tcpack_out, \
popimap_out, smtp_out, admintraffic_out, wifitraffic_out }
# Set up queues
queue standard_out bandwidth 72Kb priority 0 cbq(default, borrow)
queue dns_out bandwidth 20Kb priority 6 cbq(borrow)
queue http_out bandwidth 40Kb priority 5 cbq(borrow)
queue tcpack_out bandwidth 40Kb priority 7 cbq(borrow)
queue popimap_out bandwidth 100Kb priority 4 cbq(borrow)
queue smtp_out bandwidth 100Kb priority 3 cbq(borrow)
queue admintraffic_out bandwidth 90Kb priority 2 cbq(borrow)
queue wifitraffic_out bandwidth 50Kb priority 1 cbq(borrow)
# Set defaults.
set block-policy return
#set loginterface $ext_if
set skip on $lcl_if
scrub in on { $ext_if, $int_if }
# Allow internal interfaces to get to the internet.
nat on $ext_if from $int_ip to any -> ($ext_if)
nat on $ext_if from $adm_ip to any -> ($ext_if)
#nat-anchor "ftp-proxy/*"
# Allow local nets to FTP out. Needed??? Doesn't seem to make any difference...
#rdr-anchor "ftp-proxy/*"
#rdr on $int_if proto tcp from any to any port 21 -> 127.0.0.1 port 8021
#rdr on $adm_if proto tcp from any to any port 21 -> 127.0.0.1 port 8021
# Block everything unless later explicitly allowed.
block in
# Keep state for established connections.
pass out keep state
# Allow local nets to FTP out. Needed??? Doesn't seem to make any difference...
#anchor "ftp-proxy/*"
# Protect against IP spoofing on local network segments.
antispoof quick for { $lcl_if $int_if $adm_if }
# Block inbound traffic from IPs not valid for each interface.
block in quick on ! $int_if inet from $int_ip to any
block in quick on ! $adm_if inet from $adm_ip to any
# Nobody else is me - block attempts to make us think so.
block in quick on $int_if inet from $int_sip to any
block in quick on $adm_if inet from $adm_sip to any
# Assume administrator knows what he is doing. Not necessarily true...
pass in quick on $adm_if from $adm_ip
pass in on $ext_if inet proto tcp from any to ($ext_if) port $pop3_ports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to ($ext_if) port $imap_ports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to ($ext_if) port $smtp_ports flags S/SA keep state
pass in on $ext_if inet proto { tcp, udp } from any to ($ext_if) port $bind_ports
pass in on $ext_if inet proto tcp from any to ($ext_if) port $webi_ports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to ($ext_if) port $webo_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $pop3_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $imap_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $smtp_ports flags S/SA keep state
pass in on $int_if inet proto { tcp, udp } from any to any port $bind_ports
pass in on $int_if inet proto tcp from any to any port $webi_ports flags S/SA keep state
pass in on $int_if inet proto tcp from any to any port $webo_ports flags S/SA keep state
pass in inet proto icmp all icmp-type $icmp_types keep state
pass out on $ext_if inet proto { tcp udp } from ($ext_if) to any port $bind_ports keep state queue (dns_out, tcpack_out)
pass out on $ext_if inet proto tcp from ($ext_if) to any port $webi_ports keep state queue (http_out, tcpack_out)
pass out on $ext_if inet proto tcp from ($ext_if) to any port $mail_ports keep state queue (popimap_out, tcpack_out)
pass out on $ext_if inet proto tcp from ($ext_if) to any port $smtp_ports keep state queue (smtp_out, tcpack_out)
pass out on $ext_if inet proto tcp from ($adm_if) to any keep state queue (admintraffic_out, tcpack_out)
pass out on $ext_if inet proto tcp from ($int_if) to any keep state queue (wifitraffic_out, tcpack_out)If anybody has comments or advice on the traffic shaping, I'd appreciate a reply. Likewise, if anybody has any suggestions for where I can improve the firewall, I'd appreciate those as well.