IPFW with Jails on private IPs

P

perkypork

LAST UPDATED 20th August at 8.26am GMT
I have been able to get this to work with the below configuration in ipfw.rules. I would really appreciate it if someone could explain how this works and if what I have done is secure for my use case?

I have been using FreeBSD 10.1-RELEASE with jails (ezjails) quiet successfully for some time thanks to a number of members on this forum.

I have spent hours reading the net, man pages and the handbook. I am obviously missing something in regards to how IPFW and kernel nat work. Some sites I have looked at:
- https://github.com/nileshgr/utilities/blob/master/admin/ipfw.rules.sh
- http://nileshgr.com/2014/12/07/freebsd-ipfw-nat-jails
- https://www.freebsd.org/cgi/man.cgi?query=ipfw(8)
- https://forums.freebsd.org/threads/...exposes-local-ip-on-external-interface.52134/

I would like to run my jails on a private subnet but the jails should be able to access the net and have their own firewalls. I am going to use an nginx proxy which has a public IP which then forwards traffic off to the Jail servers so I do not need to use port forwarding.

Try as I might, I cannot seem to get IPFW using kernel nat to achieve what I want.

Below are my files (which have been updated since first post because I have been trying some new things).

/etc/rc.conf
Code: 
# Server Hostname
hostname="b5.example.com”

# Private Nets
cloned_interfaces="lo1"
ifconfig_lo1="inet 10.1.0.1/32"
ifconfig_lo1_alias0="inet 10.1.0.2/32"
ifconfig_lo1_alias1="inet 10.1.0.3/32"
ifconfig_lo1_alias2="inet 10.1.0.4/32"
ifconfig_lo1_alias3="inet 10.1.0.5/32"
ifconfig_lo1_alias4="inet 10.1.0.6/32"
ifconfig_lo1_alias5="inet 10.1.0.7/32"
ifconfig_lo1_alias6="inet 10.1.0.8/32"
ifconfig_lo1_alias7="inet 10.1.0.9/32"

ifconfig_lo1_alias8="inet 10.1.0.10/32"
ifconfig_lo1_alias9="inet 10.1.0.11/32"
ifconfig_lo1_alias10="inet 10.1.0.12/32"
ifconfig_lo1_alias11="inet 10.1.0.13/32"
ifconfig_lo1_alias12="inet 10.1.0.14/32"
ifconfig_lo1_alias13="inet 10.1.0.15/32"

# WAN
ifconfig_bce0="inet 119.111.111.111/27" # base.example.com

# Gateway
defaultrouter="119.111.111.112”

# SSH
sshd_enable="YES"

# OpenNTP
openntpd_enable="YES"
openntpd_flags="-s"

# Ezjail
ezjail_enable="YES"

# Crash dumps (AUTO/NO)
dumpdev="AUTO"

# IPFW
gateway_enable="YES"
firewall_enable="YES"
firewall_logging="YES"
firewall_nat_enable="YES"
firewall_script="/usr/local/etc/ipfw.rules"

# Syslogd
syslogd_flags="-ss"

ipfw.rules

Code: 
#!/bin/sh

######################################################
# Configuration
wif="bce0" # WAN interface
wip="119.111.111.111" # WAN IP

proxy_ports="80,443"

jif="lo1" # Jails Interface
jnet="10.1.0.0/24" # Jails Network

######################################################
# Set rules command prefix
cmd="ipfw -q add"

# Flush out the list before we begin.
ipfw -q -f flush

######################################################
# Configure NAT on WAN IP
ipfw nat 1 config ip $wip same_ports

$ Loopback interfaces
$cmd 10 allow all from any to any via lo0
$cmd 11 allow all from any to any via $jif

# NAT Rule for incoming packets
$cmd 50 nat 1 ip from any to $wip in

# Check Dynamic Rules table
$cmd 100 check-state

#$cmd 101 skipto 60000 ip4 from any to $wip recv $jif keep-state
$cmd 102 skipto 60000 ip4 from any to $wip dst-port $proxy_ports in recv $jif keep-state

# DNS for Base and Jails
$cmd 110 allow tcp from $wip to any 53 out setup keep-state
$cmd 111 allow udp from $wip to any 53 out keep-state

# Ports for Base and Jails
$cmd 120 allow tcp from $wip to any 80 out keep-state
$cmd 121 allow tcp from $wip to any 433 out keep-state

######################################################
# BASE ONLY

# Ping
$cmd 130 allow icmp from $wip to any out via $wif keep-state
$cmd 131 allow icmp from any to $wip in via $wif keep-state

# SSH
$cmd 140 allow tcp from $wip to any 22 out via $wif keep-state
$cmd 141 allow tcp from $wip to any 65222 out via $wif keep-state
$cmd 142 allow tcp from any to $wip 65222 in via $wif keep-state

# OpenNTP
$cmd 150 allow udp from $wip to any 123 out via $wif keep-state

# jail.example.com
. /usr/jails/jail.example.com/ipfw.rules

$cmd 60000 nat 1 ip4 from $jnet to any out

######################################################
# Deny Remainder and Log
$cmd deny log all from any to any
######################################################
 
Last edited:
Now that I have this working and have had a think about my infrastructure a little more, I realised that I would like to use one Public IP shared with the Host machine and the Jail network. When I do the above configuration, I think its this line
Code: 
ipfw nat 1 config ip $jwip same_ports
the base looses network connectivity. I would like the base to be able to connect to the internet on its public IP, have its own firewall and the jails have the same functionality. Is this possible or should I be using a second IP for the jails WAN IP?

To further clarify:

I want to be able to SSH to the host server on port 65222 ssh -p 65222 name@119.111.111.111

I want the proxy jail to have ports 80,443 forwarded to it so it can handle incoming HTTP requests, send them off to other jails on the 10.1.0.0/24 network and then respond back to the public internet.

I want the jails to be able to access the internet from their private network 10.1.0.0/24
 
Based on the fact that I have not had any replies, I figure I am either not asking the correct questions, I am asking a really stupid question or I posted in the incorrect forum. Not sure which. Is it possible to move a post to a new forum (myself) - firewalls?
 
You must log in or register to reply here.
Back
Top