Optimization, security & portsnap problems

S

Solaris

I have the following pf rules, and need help regarding on why portsnap in general failed to update and I am not able to connect to certain hosts (portsnap mirrors) unless pf is disabled. Also suggestions about optimization and security are welcome.

Portsnap problems:
Code: 
root@:~ # portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... failed.
Fetching snapshot tag from your-org.portsnap.freebsd.org... failed.
Fetching snapshot tag from isc.portsnap.freebsd.org... done.
Fetching snapshot metadata...

Ruleset:
Code: 
####Interfaces
ext_if="em0"
jail_if_www="lo888"
jail_if_db="lo777"
jail_if_irc="lo999"
jail_if_dns="lo111"
jail_if_psql="lo555"
jail_if_mail="lo444"
jail_if_torrent="lo222"
jail_if_voip="lo333"

####IP Assignment
IP_PUB="123.123.123.123"
IP_JAIL_WWW="10.8.8.8"
IP_JAIL_DB="10.7.7.7"
IP_JAIL_IRC="10.9.9.9"
IP_JAIL_DNS="10.1.1.1"
IP_JAIL_PSQL="10.5.5.5"
IP_JAIL_MAIL="10.4.4.4"
IP_JAIL_TORRENT="10.2.2.2"
IP_JAIL_VOIP="10.3.3.3"

###Jail Network
NET_JAIL_WWW="10.8.8.0/24"
NET_JAIL_DB="10.7.7.0/24"
NET_JAIL_IRC="10.9.9.0/24"
NET_JAIL_DNS="10.1.1.0/24"
NET_JAIL_PSQL="10.5.5.0/24"
NET_JAIL_MAIL="10.4.4.0/24"
NET_JAIL_TORRENT="10.2.2.0/24"
NET_JAIL_VOIP="10.3.3.0/24"

###PORT
PORT_WWW="{80,443,4242,44777}"
PORT_DB="{3306,9200}"
PORT_IRC="{6665,6666,6667,6668,6669,6697,7000}"
PORT_IRC_LOCAL="{8080}"
PORT_DNS="{53}"
PORT_MAIL="{993,465,25,4190,9090,9093}"
PORT_PSQL="{}"
PORT_TORRENT="{20152:65535,8081}"
PORT_VOIP="{1719:1720,3478:3479,5002:5003,5060,5070,5080,16384:32768,50888}"

set block-policy drop
set loginterface $ext_if
set optimization normal
set ruleset-optimization profile
set skip on lo0

scrub on $ext_if all reassemble tcp no-df random-id
scrub in on $ext_if all fragment reassemble no-df random-id 
scrub out on $ext_if all random-id fragment reassemble no-df
#scrub in all

# nat all www jail traffic
nat pass on $ext_if from $NET_JAIL_WWW to any -> $IP_PUB

# nat all  torrent to outside
nat pass on $ext_if from $NET_JAIL_TORRENT to any -> $IP_PUB
# redirect torrent port traffic from outbound
rdr pass on $ext_if proto tcp from any to $IP_PUB port $PORT_TORRENT -> $IP_JAIL_TORRENT
rdr pass on $ext_if proto udp from any to $IP_PUB port $PORT_TORRENT -> $IP_JAIL_TORRENT

# redirect torrent port voip from outbound
nat pass on $ext_if from $NET_JAIL_VOIP to any -> $IP_PUB
#TCP
rdr pass on $ext_if proto tcp from any to $IP_PUB port $PORT_VOIP -> $IP_JAIL_VOIP
#UDP
rdr pass on $ext_if proto udp from any to $IP_PUB port $PORT_VOIP -> $IP_JAIL_VOIP

# temp outbound access for DB
nat pass on $ext_if from $NET_JAIL_DB to any -> $IP_PUB

# temp outbound access for IRC
nat pass on $ext_if from $NET_JAIL_IRC to any -> $IP_PUB

# temp outbound access for PSQL
nat pass on $ext_if from $NET_JAIL_PSQL to any -> $IP_PUB

# temp outbound access for MAIL
nat pass on $ext_if from $NET_JAIL_MAIL to any -> $IP_PUB

# redirect mail port traffic from outbound
rdr pass on $ext_if proto tcp from any to $IP_PUB port $PORT_MAIL -> $IP_JAIL_MAIL
rdr pass on $ext_if proto udp from any to $IP_PUB port $PORT_MAIL -> $IP_JAIL_MAIL

# redirect mail port traffic from www jail
rdr pass on $jail_if_www proto tcp from any to $IP_PUB port $PORT_MAIL -> $IP_JAIL_MAIL
rdr pass on $jail_if_www proto udp from any to $IP_PUB port $PORT_MAIL -> $IP_JAIL_MAIL

# redirect irc port traffic from outbound
rdr pass on $ext_if proto tcp from any to $IP_PUB port $PORT_IRC -> $IP_JAIL_IRC

# redirect www port traffic
rdr pass on $ext_if proto tcp from any to $IP_PUB port $PORT_WWW -> $IP_JAIL_WWW


# enable DB access from www
rdr pass on $jail_if_www proto tcp from any to $IP_JAIL_DB port $PORT_DB -> $IP_JAIL_DB

# enable xml-rpc irc access from www
rdr pass on $jail_if_www proto tcp from any to $IP_JAIL_IRC port $PORT_IRC_LOCAL -> $IP_JAIL_IRC

## enable DNS access from all lumniousify network
##tcp
rdr pass on $jail_if_www proto tcp from any to $IP_JAIL_DNS port $PORT_DNS -> $IP_JAIL_DNS
rdr pass on $jail_if_db proto tcp from any to $IP_JAIL_DNS port $PORT_DNS -> $IP_JAIL_DNS
rdr pass on $jail_if_irc proto tcp from any to $IP_JAIL_DNS port $PORT_DNS -> $IP_JAIL_DNS
##udp
rdr pass on $jail_if_www proto udp from any to $IP_JAIL_DNS port $PORT_DNS -> $IP_JAIL_DNS
rdr pass on $jail_if_db proto udp from any to $IP_JAIL_DNS port $PORT_DNS -> $IP_JAIL_DNS 
rdr pass on $jail_if_irc proto udp from any to $IP_JAIL_DNS port $PORT_DNS -> $IP_JAIL_DNS 

##Jail dns
nat pass on $ext_if from $NET_JAIL_DNS to any -> $IP_PUB

#antispoof is a common special case of filtering and blocking. This mechanism protects against activity from spoofed or forged IP addresses
antispoof log for $ext_if

# activate spoofing protection for all interfaces
block in quick from urpf-failed
block in quick on $ext_if from any to 255.255.255.255
block in quick on $ext_if from urpf-failed
block in quick on $ext_if from no-route to any

# Try to block nmap scans
block in log quick on $ext_if inet proto tcp from any to any flags FUP/FUP

Running on:
Code: 
FreeBSD  9.1-STABLE FreeBSD 9.1-STABLE #0 r252471: Mon Jul  1 18:58:48 GMT 2013     root@:/usr/obj/usr/src/sys/HOST amd64
 
Hi @Solaris,

Maybe try to isolate the root cause for the failed portsnap. Is it some rule within your redirecting? Is it the web/HTTP traffic?

Some things to test:
  • Check DNS resolution with activated and deactivated pf-ruleset. Does the result change?
  • Try a manual fetch or wget with activated and deactivated pf-ruleset.
  • Use something like tcpdump -i pflog0 to see what gets blocked by your activated firewall. Use different tcpdump() parameters to fine-tune, e.g. -v or -A.
  • Try partially deactivating rules in your pf.conf to see what's the problematic portion. Use some versioning tools like rcs() so you can go back to a known state. Check in the file and play around with the different parts of the ruleset.

Maybe it has to do with your DNS-jail configuration (and DNS resolution) and not originally with pf itself. If the firewall is deactivated there will be no redirection and no DNS-jail is involved?

Just some guesses. Hope it helps.
 
Last edited by a moderator:
Found out the most probable solutions[]:
Code: 
# activate spoofing protection for all interfaces
#block in log quick from urpf-failed
block in log quick on $ext_if from any to 255.255.255.255
block in log quick on $ext_if from urpf-failed
block in log quick on $ext_if from no-route to any

# Try to block nmap scans
block in log quick on $ext_if inet proto tcp from any to any flags FUP/FUP

pass out 
pass in

This line is blocking interconnection between jail, I cannot ping, dns request into internal dns, the log
Code: 
#block in log quick from urpf-failed

tcpdump -i pflog0 -nn

Code: 
17:51:17.638800 IP 10.1.1.1 > 10.2.2.2: ICMP echo request, id 25568, seq 87, length 64
17:51:18.639777 IP 10.1.1.1 > 10.2.2.2: ICMP echo request, id 25568, seq 88, length 64
17:51:19.640798 IP 10.1.1.1 > 10.2.2.2: ICMP echo request, id 25568, seq 89, length 64
17:51:20.641802 IP 10.1.1.1 > 10.2.2.2: ICMP echo request, id 25568, seq 90, length 64

Adding "pass out" and "pass in" to the end of the rules have fixed general connectivity problems in the host or inside the jail (portsnap, etc). Are all of those 'fixes' really a proper way to fix aforementioned problems or there is better one? ?.

Here is my network configuration[]:
rc.conf :
Code: 
ifconfig_em0="DHCP"
cloned_interfaces="lo111 lo222 lo444 lo777 lo888 lo999"

ifconfig_lo111="inet 10.1.1.1 netmask 255.255.255.255"
ifconfig_lo222="inet 10.2.2.2 netmask 255.255.255.255"
ifconfig_lo444="inet 10.4.4.4 netmask 255.255.255.255"
ifconfig_lo777="inet 10.7.7.7 netmask 255.255.255.255"
ifconfig_lo888="inet 10.8.8.8 netmask 255.255.255.255"
ifconfig_lo999="inet 10.9.9.9 netmask 255.255.255.255"

ifconfig -a
Code: 
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether aa:aa:d2:aa:aa:95
        inet 123.123.123.123 netmask 0xffffff00 broadcast 123.123.123.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33152
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pfsync0: flags=0<> metric 0 mtu 1500
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        syncpeer: 0.0.0.0 maxupd: 128
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo111: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.1.1.1 netmask 0xffffffff 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo222: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.2.2.2 netmask 0xffffffff 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo444: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.4.4.4 netmask 0xffffffff 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo777: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.7.7.7 netmask 0xffffffff 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo888: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.8.8.8 netmask 0xffffffff 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo999: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.9.9.9 netmask 0xffffffff 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 
You must log in or register to reply here.
Back
Top