Hi all,
I think I need some hints and tipps from you firewall and network experts here.
Currently I have the following working scenario for IPv4:
My server has one external IPv4 address.
I'm running several jails hosting different services, each having an own 192.168.0.x IP on an internal cloned network interface.
(A jail running PostgreSQL and slapd as databases, a Jail with Prosody as chat server, a Jail with Apache + Nextcloud, a Jail with Apache + Subversion and a jail running pound as proxy to dispatch HTTP traffic between the two Apaches, since I have multiple domains.)
I'm using IPFW with in-kernel NAT to isolate the jails against each other and routing incoming traffic to the right jail.
Incoming traffic to port 5222 (prosody) is routed to the 'chat server jail', incoming traffic to port 80,443 is routed to the 'pound proxy jail' which itself forwards than to either the 'nextcloud jail' or 'subversion jail'.
The jails are isolated against each other, for example each jail can reach the postgres and slapd ports on the database jail (because the services there need a database or LDAP for authentication), but the database jail itself is not reachable from the outside.
The database jail cannot reach the nextcloud jail by itself, because there is no need for that and so on and so on...
As I said: This scenario is working fine for some years now, see my IPFW rules posted below. Pretty sure that some of the rules could be optimized, but it works and I think (/hope) I have no severy security issues with that rules. (Any hints for improvements are welcome.)
Now I (finally) want to use IPv6 in parallel.
For IPv6 I've got a whole subnet from my ISP, not only a single IPv4.
So, I think it should be possible to assign each jail an own IPv6 and set my DNS records accordingly (chat.mydomain.com to the chat jail's own IPv6, nextcloud.mydomain.com to the nextcloud jail's own IPv6 etc.).
No need to NAT and/or the pound proxy anymore.
My first question: Can I use my existing IPv4 scenario in parallel with that new IPv6 scenario? So, stick to the single external IPv4 and each jail uses an internal 192.168.0.x IPv4 on a cloned interface and using IPFW NAT + a pound proxy? And in parallel assign each jail a own IPv6 from my subnet so that I can make port 80,443 of my nextcloud jail directly accessible with its own IPv6 and also port 80,443 of my svn jail with its own IPv6?
Or is a combination of IPv6 and IPv4+NAT not possible with IPFW and I need to NAT everthing (IPv4 and IPv6) or get completely rid of IPv4?
My second question: I just started with IPv6 and tried to make as a first step the ssh server (directly running on my server, not in a jail) reachable via IPv6. THerefore I added some IPv6 statements to my IPFW rules, but must have an error here.
When I've disabled IPFW I can reach my sshd with IPv6, but if it is enabled its blocked. From that I know that the IPv6 network configuration seems to be ok and the issue must be an IPFW issue.
I really would appreciate if someone can tell me a) the right rules to open sshd for IPFW+IPv6 and b) tell me if my whole IPv4+IPv6 scenario is possible (or how it is possible).
Thanks & kind regards,
Fool
Here my current network configuration in rc.conf:
Here is my current ipfw.rules including some IPv6 attemts:
I think I need some hints and tipps from you firewall and network experts here.
Currently I have the following working scenario for IPv4:
My server has one external IPv4 address.
I'm running several jails hosting different services, each having an own 192.168.0.x IP on an internal cloned network interface.
(A jail running PostgreSQL and slapd as databases, a Jail with Prosody as chat server, a Jail with Apache + Nextcloud, a Jail with Apache + Subversion and a jail running pound as proxy to dispatch HTTP traffic between the two Apaches, since I have multiple domains.)
I'm using IPFW with in-kernel NAT to isolate the jails against each other and routing incoming traffic to the right jail.
Incoming traffic to port 5222 (prosody) is routed to the 'chat server jail', incoming traffic to port 80,443 is routed to the 'pound proxy jail' which itself forwards than to either the 'nextcloud jail' or 'subversion jail'.
The jails are isolated against each other, for example each jail can reach the postgres and slapd ports on the database jail (because the services there need a database or LDAP for authentication), but the database jail itself is not reachable from the outside.
The database jail cannot reach the nextcloud jail by itself, because there is no need for that and so on and so on...
As I said: This scenario is working fine for some years now, see my IPFW rules posted below. Pretty sure that some of the rules could be optimized, but it works and I think (/hope) I have no severy security issues with that rules. (Any hints for improvements are welcome.)
Now I (finally) want to use IPv6 in parallel.
For IPv6 I've got a whole subnet from my ISP, not only a single IPv4.
So, I think it should be possible to assign each jail an own IPv6 and set my DNS records accordingly (chat.mydomain.com to the chat jail's own IPv6, nextcloud.mydomain.com to the nextcloud jail's own IPv6 etc.).
No need to NAT and/or the pound proxy anymore.
My first question: Can I use my existing IPv4 scenario in parallel with that new IPv6 scenario? So, stick to the single external IPv4 and each jail uses an internal 192.168.0.x IPv4 on a cloned interface and using IPFW NAT + a pound proxy? And in parallel assign each jail a own IPv6 from my subnet so that I can make port 80,443 of my nextcloud jail directly accessible with its own IPv6 and also port 80,443 of my svn jail with its own IPv6?
Or is a combination of IPv6 and IPv4+NAT not possible with IPFW and I need to NAT everthing (IPv4 and IPv6) or get completely rid of IPv4?
My second question: I just started with IPv6 and tried to make as a first step the ssh server (directly running on my server, not in a jail) reachable via IPv6. THerefore I added some IPv6 statements to my IPFW rules, but must have an error here.
When I've disabled IPFW I can reach my sshd with IPv6, but if it is enabled its blocked. From that I know that the IPv6 network configuration seems to be ok and the issue must be an IPFW issue.
I really would appreciate if someone can tell me a) the right rules to open sshd for IPFW+IPv6 and b) tell me if my whole IPv4+IPv6 scenario is possible (or how it is possible).
Thanks & kind regards,
Fool
Here my current network configuration in rc.conf:
Code:
# Private IPv4 assigned by ISP
ifconfig_em0="inet A.B.C.D netmask 255.255.252.0"
# Gateway IPv4 assigned by ISP
defaultrouter="E.F.G.H"
# First IPv6 from assigned subnet by ISP
ifconfig_em0_ipv6="inet6 aaaa:bbbb:cccc:dddd::1/64"
# Additional IPv6s from assigned subnet by ISP (for jails)
ifconfig_em0_alias0="inet6 aaaa:bbbb:cccc:dddd::2/64"
ifconfig_em0_alias1="inet6 aaaa:bbbb:cccc:dddd::3/64"
[...]
# Gateway IPv6 assigned by ISP
ipv6_defaultrouter="xxxx::1%em0"
# internal local interface for jail communication
cloned_interfaces="lo1"
ifconfig_lo1="inet 192.168.0.1 netmask 255.255.255.0 mtu 1500"
icmp_drop_redirect="YES"
icmp_log_redirect="YES"
[...]
gateway_enable="YES"
firewall_enable="YES"
firewall_nat_enable="YES"
firewall_quiet="YES"
firewall_logging="YES"
firewall_script="/etc/ipfw.rules"Here is my current ipfw.rules including some IPv6 attemts:
Code:
#!/bin/sh
#################################################
# configuration
################################################
fwcmd="/sbin/ipfw -q add" # ipfw command
wan="em0" # the external interface (public internet)
wan_ip4="a.b.c.d" # the externally assigned IPv4 address
loop="lo0" # loopback interface (local network for host only)
lan="lo1" # internal interface (local network for jails)
jumpNAT="skipto 65100" # jump to NAT rule
jail_host="192.168.0.1" # jail aliases
database_jail="192.168.0.2"
proxy_jail="192.168.0.3"
chat_jail="192.168.0.4"
nextcloud_jail="192.168.0.5"
svn_jail="192.168.0.6"
################################################
# Initialization
################################################
# flush out the list before we add any rules
/sbin/ipfw -q -f flush
################################################
# Initialization for IPv4 NAT
################################################
# permit reinsertion of translated packets for further processing
/sbin/ipfw disable one_pass
# activate in-kernel NAT and define rules
/sbin/ipfw -q nat 1 config if ${wan} same_ports unreg_only reset \
redirect_port tcp ${proxy_jail}:80 80 \
redirect_port tcp ${proxy_jail}:443 443 \
redirect_port tcp ${chat_jail}:5222 5222
################################################
# LOCAL traffic (rules 0-499)
################################################
# ALLOW anything within the loopback interface
${fwcmd} 00001 allow all from any to any via ${loop}
# ALLOW each jail to reach itself without restriction
${fwcmd} 00010 allow all from ${jail_host} to ${jail_host} via ${lan}
${fwcmd} 00011 allow all from ${database_jail} to ${database_jail} via ${lan}
${fwcmd} 00012 allow all from ${proxy_jail} to ${proxy_jail} via ${lan}
${fwcmd} 00013 allow all from ${chat_jail} to ${chat_jail} via ${lan}
${fwcmd} 00014 allow all from ${nextcloud_jail} to ${nextcloud_jail} via ${lan}
${fwcmd} 00015 allow all from ${svn_jail} to ${svn_jail} via ${lan}
################################################
# NAT and existing entries in dynamic rules table
# (rules 500-999)
################################################
# reassemble inbound packets
${fwcmd} 00500 reass all from any to any in
# NAT any inbound packets
${fwcmd} 00501 nat 1 ip from any to any in via ${wan}
# allow packets having an existing entry in the dynamic rules table
${fwcmd} 00502 check-state
################################################
# OUTBOUND traffic (rules 1000-9999)
################################################
# ALLOW access to public DNS (see /etc/resolve.conf)
${fwcmd} 01000 ${jumpNAT} tcp from any to a.b.c.d 53 out via ${wan} setup keep-state
${fwcmd} 01001 ${jumpNAT} udp from any to a.b.c.d 53 out via ${wan} keep-state
${fwcmd} 01002 ${jumpNAT} tcp from any to a.b.c.d 53 out via ${wan} setup keep-state
${fwcmd} 01003 ${jumpNAT} udp from any to a.b.c.d 53 out via ${wan} keep-state
${fwcmd} 01004 ${jumpNAT} tcp from any to ${wan_ip4} 443 out via ${wan} setup keep-state
${fwcmd} 01005 ${jumpNAT} udp from any to ${wan_ip4} 443 out via ${wan} keep-state
# ALLOW access to OpenNTP (see /usr/local/etc/ntpd.conf)
${fwcmd} 01010 ${jumpNAT} udp from any to any 123 out via ${wan} keep-state
# ALLOW nextcloud jail some outbound traffic
# send mail
${fwcmd} 01050 ${jumpNAT} tcp from ${nextcloud_jail} to A.B.C.D 465,993 out via ${wan} setup keep-state
# use HTTP / HTTPS connections
${fwcmd} 01051 ${jumpNAT} tcp from ${nextcloud_jail} to any 80,443 out via ${wan} setup keep-state
# ALLOW svn jail some outbound traffic
# send mail
${fwcmd} 01080 ${jumpNAT} tcp from ${svn_jail} to A.B.C.D 465,993 out via ${wan} setup keep-state
# ALLOW traffic to services between jails (note: no NAT required)
# to database jail
${fwcmd} 01100 allow tcp from ${jail_host} to ${database_jail} 389,636,5432 out via ${lan} setup keep-state
${fwcmd} 01101 allow tcp from ${chat_jail} to ${database_jail} 389 out via ${lan} setup keep-state
${fwcmd} 01102 allow tcp from ${nextcloud_jail} to ${database_jail} 636,5432 out via ${lan} setup keep-state
${fwcmd} 01103 allow tcp from ${svn_jail} to ${database_jail} 636 out via ${lan} setup keep-state
# to proxy jail
${fwcmd} 01110 allow tcp from ${jail_host} to ${proxy_jail} 80,443 out via ${lan} setup keep-state
# to chat jail
${fwcmd} 01120 allow tcp from ${jail_host} to ${chat_jail} 5222 out via ${lan} setup keep-state
${fwcmd} 01121 allow tcp from ${svn_jail} to ${chat_jail} 5222 out via ${lan} setup keep-state
# to nextcloud jail
${fwcmd} 01130 allow tcp from ${jail_host} to ${nextcloud_jail} 80,443 out via ${lan} setup keep-state
${fwcmd} 01131 allow tcp from ${proxy_jail} to ${nextcloud_jail} 80,443 out via ${lan} setup keep-state
# to svn jail
${fwcmd} 01140 allow tcp from ${jail_host} to ${svn_jail} 80,443 out via ${lan} setup keep-state
${fwcmd} 01141 allow tcp from ${proxy_jail} to ${svn_jail} 80,443 out via ${lan} setup keep-state
# ALLOW each jail to reach the host's syslogd and exim (note: no NAT required)
${fwcmd} 01200 allow udp from 192.168.0.0/24 to ${jail_host} 514 out via ${lan} keep-state
${fwcmd} 01201 allow tcp from 192.168.0.0/24 to ${jail_host} 25 out via ${lan} setup keep-state
# ALLOW root user to reach anything - this applies also for jails!
# (important for updates etc.)
${fwcmd} 01300 ${jumpNAT} tcp from me to any out via ${wan} setup keep-state uid root
${fwcmd} 01301 ${jumpNAT} udp from me to any out via ${wan} keep-state uid root
${fwcmd} 01302 ${jumpNAT} icmp from me to any out via ${wan} keep-state
# same for ipv6
${fwcmd} 01310 allow tcp from me6 to any out via ${wan} setup keep-state uid root
${fwcmd} 01311 allow udp from me6 to any out via ${wan} keep-state uid root
${fwcmd} 01312 allow ipv6-icmp from me6 to any out via ${wan} keep-state
# DENY and log all other outbound connections
${fwcmd} 09999 deny log all from any to any out via ${wan}
################################################
# INBOUND traffic (rules 10000-19999)
################################################
# DENY all inbound traffic from non-routable reserved address spaces
${fwcmd} 10001 deny all from 192.168.0.0/16 to any in via ${wan} #RFC 1918 private IP
${fwcmd} 10002 deny all from 172.16.0.0/12 to any in via ${wan} #RFC 1918 private IP
${fwcmd} 10003 deny all from 10.0.0.0/8 to any in via ${wan} #RFC 1918 private IP
${fwcmd} 10004 deny all from 127.0.0.0/8 to any in via ${wan} #loopback
${fwcmd} 10005 deny all from 0.0.0.0/8 to any in via ${wan} #loopback
${fwcmd} 10006 deny all from 169.254.0.0/16 to any in via ${wan} #DHCP auto-config
${fwcmd} 10007 deny all from 192.0.2.0/24 to any in via ${wan} #reserved for docs
${fwcmd} 10008 deny all from 204.152.64.0/23 to any in via ${wan} #Sun cluster interconnect
${fwcmd} 10009 deny all from 224.0.0.0/3 to any in via ${wan} #Class D & E multicast
# DENY public pings (only allow icmptype 0 if we ping stuff ourself)
${fwcmd} 10010 deny icmp from any to any in via ${wan} not icmptypes 0
${fwcmd} 10011 deny ipv6-icmp from any to any in via ${wan} not icmp6types 0
# DENY ident/ noise from routers
${fwcmd} 10020 deny tcp from any to any 113 in via ${wan}
${fwcmd} 10021 deny udp from any to any 520 in via ${wan}
# DENY all Netbios services
${fwcmd} 10030 deny tcp from any to any 137 in via ${wan}
${fwcmd} 10031 deny tcp from any to any 138 in via ${wan}
${fwcmd} 10032 deny tcp from any to any 139 in via ${wan}
${fwcmd} 10033 deny tcp from any to any 81 in via ${wan}
# DENY fragments
${fwcmd} 10040 deny all from any to any frag in via ${wan}
# DENY ACK packets that didn't match the dynamic rule table
${fwcmd} 10041 deny tcp from any to any established in via ${wan}
# DENY broadcasts and multicasts
${fwcmd} 10050 deny ip from any to 255.255.255.255
${fwcmd} 10051 deny ip from any to 224.0.0.0/24 in
# DENY spoofing from outside
${fwcmd} 10060 deny ip from any to any not antispoof in via ${wan}
${fwcmd} 10061 deny all from any to 127.0.0.0/8 in via ${wan}
${fwcmd} 10062 deny all from any to ::1 in via ${wan}
${fwcmd} 10063 deny all from ::1 to any via ${wan}
# DENY specific IPs
# currently none
# number 00015 reserved for bans from fail2ban (-> TBD in the future)
# ${fwcmd} 00015 deny all from <ban_ip> to me in via ${wan}
# ALLOW inbound SSH connections to real sshd
${fwcmd} 11000 set 31 allow log tcp from any to me 2244 in via ${wan} setup limit src-addr 3
${fwcmd} 11001 set 31 allow log tcp from any to me6 2244 in via ${wan} setup limit src-addr 3
# ALLOW inbound SSH connections to tar pit sshd
${fwcmd} 11002 allow tcp from any to me 2222 in via ${wan} setup keep-state
${fwcmd} 11003 allow tcp from any to me6 2222 in via ${wan} setup keep-state
# ALLOW inbound connections to services on jails
# Apache routed via proxy_jail
${fwcmd} 12000 ${jumpNAT} tcp from any to ${proxy_jail} 80,443 in via ${wan} setup keep-state
# Prosody in chat_jail
${fwcmd} 12100 ${jumpNAT} tcp from any to ${chat_jail} 5222 in via ${wan} setup keep-state
# DENY and log all other incoming connections
${fwcmd} 19999 deny log all from any to any in via ${wan}
################################################
## FINAL rules (>=65000)
################################################
# DENY and LOG all uncaptured messages on ANY interface
${fwcmd} 65000 deny log all from any to any
# ALLOW skipto location for outbound statefule rules
${fwcmd} 65100 nat 1 ip from 192.168.0.0/24 to any out via ${wan}
${fwcmd} 65101 allow ip from any to any
