IPFW Set up IPFW with Jails

Hey,

I am stuck trying to set up ipfw(8), my current config is:

1 Virtual Machine running pfSense (2 NICs - Firewall working fine).
1 Virtual Machine with FreeBSD 10.1 using 2 jails running Bind (1 NIC - willing limit ports from host and jails).

Observation: Servers are in an DMZ zone.

Not using ipfw:
Code:
firewall_enable="NO"

Everything works fine. When I try setup IPFW on the host and inside jails everything stop working. I was following this guide:

FreeBSD Manual Chapter 30. Firewalls.

Then I tried to set up only the host first, this is my ipfw.rules:
Code:
#!/bin/sh
# Flush out the list before we begin
ipfw -q -f flush

# Set rules command prefix
cmd="ipfw -q add"
$DMZ="em0"  # interface name of NIC attached to Internet


# Change xl0 to LAN NIC interface name
# $cmd 00010 allow all from any to any via $DMZ

# No restrictions on Loopback Interface
$cmd 00020 allow all from any to any via lo0

# Setting ICMP
# 0 = Echo Reply
# 3 = Destination Unreachable
# 8 = Echo Request
# 11 = TTL
# 12 = Problem Header Parameter
# 13 = Timestamp Request
# 14 = Timestamp Reply

# Send Ping
$cmd 00030 allow icmp from any to any icmtypes 0 in
$cmd 00040 allow icmp from any to any icmtypes 8 out

# Respond Ping
$cmd 00050 deny icmp from any to any icmtypes 0 out
$cmd 00060 deny icmp from any to any icmtypes 8 in

# Request traceroute
$cmd 00070 allow icmp from any to any icmtypes 11 in

# Respond traceroute
$cmd 00080 deny icmp from any to any icmtypes 11 out

$cmd 00090 check-state

# DMZ -> Internet
# Allow access to public DNS
# Replace x.x.x.x with the IP address of a public DNS server
# and repeat for each DNS server in /etc/resolv.conf
$cmd 00100 allow tcp from 192.168.0.10 to 192.168.0.13 53 out via $DMZ setup keep-state
$cmd 00110 allow tcp from 192.168.0.10 to 208.67.222.222 53 out via $DMZ setup keep-state
$cmd 00120 allow tcp from 192.168.0.10 to 208.67.220.220 53 out via $DMZ setup keep-state
$cmd 00130 allow udp from 192.168.0.10 to 192.168.0.13 53 out via $DMZ keep-state
$cmd 00140 allow udp from 192.168.0.10 to 208.67.222.222 53 out via $DMZ keep-state
$cmd 00150 allow udp from 192.168.0.10 to 208.67.220.220 53 out via $DMZ keep-state

# Allow outbound NTP
$cmd 00160 allow udp from 192.168.0.10 to any 37 out via $DMZ setup keep-state

# Allow access to ISP's DHCP server for cable/DSL configurations.
# Use the first rule and check log for IP address.
# Then, uncomment the second rule, input the IP address, and delete the first rule
# $cmd 00170 allow log udp from any to any 67 out via $DMZ keep-state
# $cmd 00180 allow udp from any to x.x.x.x 67 out via $DMZ keep-state

# Allow outbound HTTP and HTTPS and FTP connections
$cmd 00200 allow tcp from 192.168.0.10 to any 80 out via $DMZ setup keep-state
$cmd 00210 allow tcp from 192.168.0.10 to any 443 out via $DMZ setup keep-state
# $cmd 00220 allow tcp from 192.168.0.10 to any 21 out via $DMZ setup keep-state

# Allow outbound email connections
# SMTP (TCP/UDP) - POP3 - IMAP (TCP/UDP) - SMTPS - IMAPS - POP3S
# $cmd 00300 allow tcp from 192.168.0.10 to any 25 out via $DMZ setup keep-state
# $cmd 00310 allow udp from 192.168.0.10 to any 25 out via $DMZ setup keep-state
# $cmd 00320 allow tcp from 192.168.0.10 to any 110 out via $DMZ setup keep-state
# $cmd 00330 allow tcp from 192.168.0.10 to any 143 out via $DMZ setup keep-state
# $cmd 00340 allow udp from 192.168.0.10 to any 143 out via $DMZ setup keep-state
# $cmd 00350 allow tcp from 192.168.0.10 to any 465 out via $DMZ setup keep-state
# $cmd 00360 allow tcp from 192.168.0.10 to any 993 out via $DMZ setup keep-state
# $cmd 00370 allow tcp from 192.168.0.10 to any 995 out via $DMZ setup keep-state


# Allow outbound SSH
# $cmd 00400 allow tcp from 192.168.0.10 to any 22 out via $DMZ setup keep-state

# Deny and Log all other Outbound connections
$cmd 00500 deny log all from any to any out via $DMZ


# Internet -> DMZ
# Deny all inbound traffic from non-routable reserved address spaces
$cmd 00600 deny all from 192.168.0.0/16 to any in via $DMZ  # RFC 1918 private IP
$cmd 00610 deny all from 172.16.0.0/12 to any in via $DMZ  # RFC 1918 private IP
$cmd 00620 deny all from 10.0.0.0/8 to any in via $DMZ  # RFC 1918 private IP
$cmd 00630 deny all from 127.0.0.0/8 to any in via $DMZ  # Loopback
$cmd 00640 deny all from 0.0.0.0/8 to any in via $DMZ  # Loopback
$cmd 00650 deny all from 169.254.0.0/16 to any in via $DMZ  # DHCP auto-config
$cmd 00660 deny all from 192.0.2.0/24 to any in via $DMZ  # Reserved for Docs
$cmd 00670 deny all from 204.152.64.0/23 to any in via $DMZ  # Sun Cluster Interconnect
$cmd 00680 deny all from 224.0.0.0/3 to any in via $DMZ  # Class D & E Multicast

# Deny ident
$cmd 00700 deny tcp from any to any 113 in via $DMZ

# Deny all Netbios Services.
$cmd 00710 deny tcp from any to any 137 in via $DMZ
$cmd 00720 deny tcp from any to any 138 in via $DMZ
$cmd 00730 deny tcp from any to any 139 in via $DMZ
$cmd 00740 deny tcp from any to any 81 in via $DMZ

# Deny fragments
$cmd 00750 deny all from any to any frag in via $DMZ

# Deny ACK packets that did not match the dynamic rule table
$cmd 00760 deny tcp from any to any established in via $DMZ

# Allow traffic from ISP's DHCP server.
# Replace x.x.x.x with the same IP address used in rule 00120.
# $cmd 00770 allow udp from any to 192.168.0.1 67 in via $DMZ keep-state

# Server Services
# Allow DNS connections to Internal DNS Server
# $cmd 00800 allow tcp from any to me 53 in via $DMZ setup limit src-addr 2
# $cmd 00810 allow udp from any to me 53 in via $DMZ setup limit src-addr 2

# Allow HTTP and FTP connections to Internal Web Server
# $cmd 00900 allow tcp from any to me 80 in via $DMZ setup limit src-addr 2
# $cmd 00910 allow tcp from any to me 443 in via $DMZ setup limit src-addr 2
# $cmd 00920 allow tcp from any to me 21 in via $DMZ setup limit src-addr 2

# Allow outbound email connections
# SMTP (TCP/UDP) - POP3 - IMAP (TCP/UDP) - SMTPS - IMAPS - POP3S
# $cmd 01000 allow tcp from any to me 25 in via $DMZ setup limit src-addr 2
# $cmd 01010 allow udp from any to me 25 in via $DMZ setup limit src-addr 2
# $cmd 01020 allow tcp from any to me 110 in via $DMZ setup limit src-addr 2
# $cmd 01030 allow tcp from any to me 143 in via $DMZ setup limit src-addr 2
# $cmd 01040 allow udp from any to me 143 in via $DMZ setup limit src-addr 2
# $cmd 01050 allow tcp from any to me 465 in via $DMZ setup limit src-addr 2
# $cmd 01060 allow tcp from any to me 993 in via $DMZ setup limit src-addr 2
# $cmd 01070 allow tcp from any to me 995 in via $DMZ setup limit src-addr 2

# Allow Inbound SSH connections
# $cmd 01100 allow tcp from any to me 22 in via $DMZ setup limit src-addr 2

# Reject and Log All other Incoming connections
$cmd 01200 deny log all from any to any in via $DMZ

# Everything else is denied and logged
$cmd 01210 deny log all from any to any

My setup in /etc/rc.conf:
Code:
...
firewall_enable="NO"
firewall_script="/etc/ipfw.rules"
firewall_logging="NO"
...

So for testing the firewall I do:
/etc/rc.d/ipfw onestart
With the previous script ntp doesn't work, and I can't browse using links or ping internet domains. I can ping my local network and internet domains by IP.

If I set:
Code:
# Change xl0 to LAN NIC interface name
# $cmd 00010 allow all from any to any via $DMZ
To
Code:
# Change xl0 to LAN NIC interface name
$cmd 00010 allow all from any to any via $DMZ
Then everything works.

By the way, seems to me everything become allowed since this is on top of rules and the rest of script become useless. Is that right?

My goal is:

Host has the following ports open:
53, 80, 443, 123 or 37 (not sure which one is for ntp), ICMP ping for test connections

Jails:
53, 123 or 37 (not sure which one is for ntp), ICMP ping for test connections

What am I missing?
 
Since the FreeBSD "handbook" manual have no instructions about how to deal with firewalls and jails I'm searching about how to get my setup working, so, someone would send me an updated how to or guide to read and get the concepts please?

Mostly of links I have found is about PF and outdated...
 
The firewall (regardless if it's PF or IPFW) needs to be configured on the host. With regular jails you cannot change any of the network settings (including firewalls). So the only way to firewall the jail is by firewalling the host.
 
I'm changing the script trying set everything on host (instead enable firewall on host and per jail like: Host /etc/rc.conf - Jail 1 /usr/jails/server1/etc/rc.conf - Jail 2 /usr/jails/server2/etc/rc.conf), by the way, I'm curious about whats the meaning of:

With regular jails you cannot change any of the network settings (including firewalls)

I followed entire handbook, doing the commands:

make buildworld
make installworld

But, I have not used the easy jail, my /etc/jail.conf:
Code:
server1 {
        path = /usr/jails/server1;
        mount.devfs;
        host.hostname = server1.mydomain.com;
        ip4addr = 192.168.0.11;
        interface = em0;
        exec.start = "/bin/sh /etc/rc";
        exec.stop = "/bin/sh /etc/rc.shutdown";
}


server2 {
        path = /usr/jails/server2;
        mount.devfs;
        host.hostname = server2.mydomain.com;
        ip4addr = 192.168.0.13;
        interface = em0;
        exec.start = "/bin/sh /etc/rc";
        exec.stop = "/bin/sh /etc/rc.shutdown";
}
This should be a regular jail? And what are other jail types?

I have read something about VLAN and make clones from em0, but didn't get the entire idea how this works.
 
VLANs are for segmenting networks, you may have seen VIMAGE. With VIMAGE the whole network stack is also separated and would allow a firewall inside a jail. This feature requires a custom kernel and is still very much experimental. It's usually not needed though, for most situations a regular jail(8) will do fine.

I don't use IPFW but I think your firewall rules don't allow traffic in/out of the IP addresses from the jails. That's probably why everything stops. Judging by the IP addresses and interfaces, everything is on the same network. Then simply imagine the jail as a separate machine with its own IP address and add the necessary rules on the host's firewall.
 
I have rewritten the rules on the host for the setup I'm looking for and now I'm getting this message:

Code:
Jan 7 00:04:40 host ntpd[586]: sendto(187.103.96.33) (fd=22): Permission denied
Jan 7 00:04:52 host ntpd[586]: sendto(187.103.96.32) (fd=22): Permission denied
Jan 7 00:04:54 host ntpd[586]: sendto(200.160.7.193) (fd=22): Permission denied
Jan 7 00:04:55 host ntpd[586]: sendto(146.164.53.65) (fd=22): Permission denied
Jan 7 00:04:57 host ntpd[586]: sendto(200.192.232.8) (fd=22): Permission denied
Jan 7 00:05:00 host ntpd[586]: sendto(187.49.33.13) (fd=22): Permission denied

And can not browse with links or resolve internet domain names, but I can [COLOR=#59b300]ping[/COLOR] internet IP addresses.

Do I need to use gateway or set something on jails to enable forward from host to jail?

Note 1: I already have set for test on the firewall:

Code:
allow tcp from any to any 53 out via $DMZ setup keep-state
allow udp from any to any 53 out via $DMZ setup keep-state

allow tcp from any to any 53 in via $DMZ setup keep-state
allow udp from any to any 53 in via $DMZ setup keep-state

Note 2: Removing the setup keep-state, Host can solve names using [COLOR=#59b300]ping[/COLOR], but the way, ntpd still has permission denied.

Example:


Code:
allow tcp from any to any 53 out via $DMZ
allow udp from any to any 53 out via $DMZ

allow tcp from any to any 53 in via $DMZ
allow udp from any to any 53 in via $DMZ
 
NTP uses port UDP/123 which isn't allowed anywhere in your firewall rules. NTP isn't going to work inside a jail anyway. The jail gets the time from the host and can't be set from inside a jail. So run your NTP on the host if you require it.
 
Back
Top