Hello,
sure, I'll show you my ipfw rules. I'm also open for any hints etc. if there is something to improve that doesn't have to do anything with my issue.
It is a long time ago that I've created these jails and I'm not sure what documentation / guideline I was following. But from a quick look into the handbook I would say yes, these are VNET jails:
Each jail has an own internal IP address and is isolated from all other jails. TCP based access is strictly controlled by ipfw rules.
The idea why I'm doing this is for an additional layer of security. E.g. if someone is able to break into my Nextcloud, the attacker (at least in theory) has only access to nextcloud's files but not to other services on my server. My jails are managed via iocage.
My setup is as followed:
I have a "database jail" which runs PostgreSQL and OpenLDAP (user authentication), this jail is not reachable from the outside / public.
I have a "nextcloud jail" running Nextcloud on Apache. Nextcloud is able to access (via TCP) PostgreSQL and OpenLDAP in the "database jail" for user authentication and storing its (meta) data.
I also have a "svn jail" running a private Subversion Repository with webdav on Apache, this jail also can reach the OpenLDAP in the "database jail" for user auth via TCP.
Incoming http(s) requests to my server are first routet to my forth jail, the "proxy jail". Here I have a pound proxy running. Depending on the hostname/url of the incoming http(s) request,
pount routes the request to either the nextcloud or the svn jail.
For testing purposes I already have disabled the proxy jail and made the nextcloud jail reachable directly, without pound in the middle.
That didn't fix the problem, so I would assume that the issue has nothing to do with my "proxy jail"...
Here are my rules:
sh:
#!/bin/sh
#################################################
# configuration
################################################
fwcmd="/sbin/ipfw -q add" # ipfw command
wan="em0" # the external interface (public internet)
loop="lo0" # loopback interface (local network for host only)
lan="lo1" # internal interface (local network for jails)
jumpNAT="skipto 65100" # jump to NAT rule
jail_host="192.168.0.1" # jail aliases
database_jail="192.168.0.2"
proxy_jail="192.168.0.3"
nextcloud_jail="192.168.0.4"
svn_jail="192.168.0.5"
################################################
# INIT
################################################
# flush out the list before we add any rules
/sbin/ipfw -q -f flush
# activate in-kernel NAT and define rules
/sbin/ipfw disable one_pass
/sbin/ipfw -q nat 1 config if ${wan} same_ports unreg_only reset \
redirect_port tcp ${proxy_jail}:80 80 \
redirect_port tcp ${proxy_jail}:443 443
################################################
# LOCAL traffic (rules 0-499)
################################################
# ALLOW anything within the loopback interface
${fwcmd} 00001 allow all from any to any via ${loop}
# ALLOW each jail to reach itself without restriction
${fwcmd} 00010 allow all from ${jail_host} to ${jail_host} via ${lan}
${fwcmd} 00011 allow all from ${database_jail} to ${database_jail} via ${lan}
${fwcmd} 00012 allow all from ${proxy_jail} to ${proxy_jail} via ${lan}
${fwcmd} 00013 allow all from ${nextcloud_jail} to ${nextcloud_jail} via ${lan}
${fwcmd} 00014 allow all from ${svn_jail} to ${svn_jail} via ${lan}
################################################
# NAT and existing entries in dynamic rules table
# (rules 500-999)
################################################
# reassemble inbound packets
${fwcmd} 00500 reass all from any to any in
# NAT any inbound packets
${fwcmd} 00501 nat 1 ip from any to any in via ${wan}
# allow packets having an existing entry in the dynamic rules table
${fwcmd} 00502 check-state
################################################
# OUTBOUND traffic (rules 1000-9999)
################################################
# ALLOW access to public DNS (see /etc/resolve.conf)
${fwcmd} 01000 ${jumpNAT} tcp from any to <dns1> 53 out via ${wan} setup keep-state
${fwcmd} 01001 ${jumpNAT} udp from any to <dns1> 53 out via ${wan} keep-state
${fwcmd} 01002 ${jumpNAT} tcp from any to <dns2> 53 out via ${wan} setup keep-state
${fwcmd} 01003 ${jumpNAT} udp from any to <dns2> 53 out via ${wan} keep-state
# ALLOW access to OpenNTP (see /usr/local/etc/ntpd.conf)
${fwcmd} 01010 ${jumpNAT} udp from any to any 123 out via ${wan} keep-state
# ALLOW nextcloud jail some outbound traffic:
# send mails
${fwcmd} 01050 ${jumpNAT} tcp from ${nextcloud_jail} to <mailserver> 465,993 out via ${wan} setup keep-state
# use HTTP / HTTPS connections
${fwcmd} 01051 ${jumpNAT} tcp from ${nextcloud_jail} to any 80,443 out via ${wan} setup keep-state
# ALLOW svn jail some outbound traffic
# send mails
${fwcmd} 01080 ${jumpNAT} tcp from ${svn_jail} to <mailserver> 465,993 out via ${wan} setup keep-state
# ALLOW traffic to services between jails (note: no NAT required)
# to database jail
${fwcmd} 01100 allow tcp from ${jail_host} to ${database_jail} 389,636,5432 out via ${lan} setup keep-state
${fwcmd} 01102 allow tcp from ${nextcloud_jail} to ${database_jail} 636,5432 out via ${lan} setup keep-state
${fwcmd} 01104 allow tcp from ${svn_jail} to ${database_jail} 636 out via ${lan} setup keep-state
# to proxy jail
${fwcmd} 01110 allow tcp from ${jail_host} to ${proxy_jail} 80,443 out via ${lan} setup keep-state
# to nextcloud jail
${fwcmd} 01130 allow tcp from ${jail_host} to ${nextcloud_jail} 80,443 out via ${lan} setup keep-state
${fwcmd} 01131 allow tcp from ${proxy_jail} to ${nextcloud_jail} 80,443 out via ${lan} setup keep-state
# to svn jail
${fwcmd} 01150 allow tcp from ${jail_host} to ${svn_jail} 80,443 out via ${lan} setup keep-state
${fwcmd} 01151 allow tcp from ${proxy_jail} to ${svn_jail} 80,443 out via ${lan} setup keep-state
# ALLOW each jail to reach the host's syslogd and exim (note: no NAT required)
${fwcmd} 01200 allow udp from 192.168.0.0/24 to ${jail_host} 514 out via ${lan} keep-state
${fwcmd} 01201 allow tcp from 192.168.0.0/24 to ${jail_host} 25 out via ${lan} setup keep-state
# ALLOW root user to reach anything
# (important for updates etc.)
${fwcmd} 01300 ${jumpNAT} tcp from me to any out via ${wan} setup keep-state uid root
${fwcmd} 01301 ${jumpNAT} udp from me to any out via ${wan} keep-state uid root
${fwcmd} 01302 ${jumpNAT} icmp from me to any out via ${wan} keep-state
# same for ipv6
${fwcmd} 01310 allow tcp from me6 to any out via ${wan} setup keep-state uid root
${fwcmd} 01311 allow udp from me6 to any out via ${wan} keep-state uid root
#${fwcmd} 01312 allow ipv6-icmp from me6 to any out via ${wan} keep-state
# DENY and log all other outbound connections
${fwcmd} 09999 deny log all from any to any out via ${wan}
################################################
# INBOUND traffic (rules 10000-19999)
################################################
# DENY all inbound traffic from non-routable reserved address spaces
${fwcmd} 10001 deny all from 192.168.0.0/16 to any in via ${wan} #RFC 1918 private IP
${fwcmd} 10002 deny all from 172.16.0.0/12 to any in via ${wan} #RFC 1918 private IP
${fwcmd} 10003 deny all from 10.0.0.0/8 to any in via ${wan} #RFC 1918 private IP
${fwcmd} 10004 deny all from 127.0.0.0/8 to any in via ${wan} #loopback
${fwcmd} 10005 deny all from 0.0.0.0/8 to any in via ${wan} #loopback
${fwcmd} 10006 deny all from 169.254.0.0/16 to any in via ${wan} #DHCP auto-config
${fwcmd} 10007 deny all from 192.0.2.0/24 to any in via ${wan} #reserved for docs
${fwcmd} 10008 deny all from 204.152.64.0/23 to any in via ${wan} #Sun cluster interconnect
${fwcmd} 10009 deny all from 224.0.0.0/3 to any in via ${wan} #Class D & E multicast
# DENY public pings (only allow icmptype 0 if we ping stuff ourself)
${fwcmd} 10010 deny icmp from any to any in via ${wan} not icmptypes 0
${fwcmd} 10011 deny ipv6-icmp from any to any in via ${wan} not icmp6types 0
# DENY ident/ noise from routers
${fwcmd} 10020 deny tcp from any to any 113 in via ${wan}
${fwcmd} 10021 deny udp from any to any 520 in via ${wan}
# DENY all Netbios services
${fwcmd} 10030 deny tcp from any to any 137 in via ${wan}
${fwcmd} 10031 deny tcp from any to any 138 in via ${wan}
${fwcmd} 10032 deny tcp from any to any 139 in via ${wan}
${fwcmd} 10033 deny tcp from any to any 81 in via ${wan}
# DENY fragments
${fwcmd} 10040 deny all from any to any frag in via ${wan}
# DENY ACK packets that didn't match the dynamic rule table
${fwcmd} 10041 deny tcp from any to any established in via ${wan}
# DENY broadcasts and multicasts
${fwcmd} 10050 deny ip from any to 255.255.255.255
${fwcmd} 10051 deny ip from any to 224.0.0.0/24 in
# DENY spoofing from outside
${fwcmd} 10060 deny ip from any to any not antispoof in via ${wan}
${fwcmd} 10061 deny all from any to 127.0.0.0/8 in via ${wan}
${fwcmd} 10062 deny all from any to ::1 in via ${wan}
${fwcmd} 10063 deny all from ::1 to any via ${wan}
# number 00015 reserved for bans from fail2ban
# ${fwcmd} 00015 deny all from <ban_ip> to me in via ${wan}
# ALLOW inbound SSH connections
${fwcmd} 11000 set 31 allow log tcp from any to me 22 in via ${wan} setup limit src-addr 3
# ALLOW inbound connections to services on jails
# http(s) requests routed via proxy_jail to nextcloud or svn jail
${fwcmd} 12000 ${jumpNAT} tcp from any to ${proxy_jail} 80,443 in via ${wan} setup keep-state
# DENY and log all other incoming connections
${fwcmd} 19999 deny log all from any to any in via ${wan}
################################################
## FINAL rules (>=65000)
################################################
# DENY and LOG all uncaptured messages on ANY interface
${fwcmd} 65000 deny log all from any to any
# ALLOW skipto location for outbound statefule rules
${fwcmd} 65100 nat 1 ip from 192.168.0.0/24 to any out via ${wan}
${fwcmd} 65101 allow ip from any to any
Kind regards,
Fool
