I cannot limit cross-talking between jails.
Host (serv) is sitting on a network, and is supposed to serve three ips. 2222 is opened for ssh on serv's ip. Everything else is blocked by "block", except for common "skip on lo0".
Two jails (t1 and t2) are assigned ips on em0 by ezjail.
t2 has a web server.
t1 is able to connect to this server! Why?
pf.conf is this:
rc.conf is this:
Output from ifconfig looks like this:
Output from "netstat -r" is following:
Do I interpret netstat correctly, if I'll say that skipping of a block happens due to jails ips magically going through lo0, on which there is a skip?
The second question is, how to set two jails on the host so that, one may talk to another over one port, with everything else blocked?
It seems to me that most common examples are such that pf cannot block unauthorized crosstalk between jails. Please, show me example otherwise.
Put on your adversarial mood, imagine that you busted process in a jail, cannot break out of the jail, yet, try to connect to neighboring jail. If you can connect, it is the issue I am talking about, if you cannot connect, can you share your network, pf and jail settings.
I tried to put jails on lo1, but had to add skip on it as well, which allows authorized cross-talk. Experiments with giving jails ips on tap's, also added ip to lo0 relationship in netstat.
What am I missing here?
Thank you, and ... he-e-elp!
PS all of this has been tested on FreeBSD 11.0.
[UPDATE] in this thread below we have one solution that involves keeping every single jail on its own loopback, having filtering on loopbacks and skip on host's lo0. It is possible to filter inter-jails communication in this setting. The caveate is that jail cannot have external ip attached to it. So, to serve traffic from outside, port(s) need to be pf-ed inside.
Hope, this will help someone.
Do leave comments, suggestions, questions.
Host (serv) is sitting on a network, and is supposed to serve three ips. 2222 is opened for ssh on serv's ip. Everything else is blocked by "block", except for common "skip on lo0".
Two jails (t1 and t2) are assigned ips on em0 by ezjail.
t2 has a web server.
t1 is able to connect to this server! Why?
pf.conf is this:
Code:
set skip on lo0
ext_if = "em0"
serv = "192.168.0.10"
t1 = "192.168.0.11"
t2 = "192.168.0.12"
# removing 192.168.0.0/16 as we face such network
table <martians> const { 127.0.0.0/8, 172.16.0.0/12, \
10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
0.0.0.0/8, 240.0.0.0/4 }
block
block drop in quick on $ext_if from <martians> to any
block drop out quick on $ext_if from any to <martians>
# Allow ssh connections to serv
pass in inet proto tcp from any to $serv port { 2222 }
antispoof for $ext_if
rc.conf is this:
Code:
hostname="serv"
defaultrouter="192.168.0.1"
ifconfig_em0="inet 192.168.0.10 netmask 255.255.255.0"
gateway_enable="YES"
pf_enable="YES"
sshd_enable="YES"
Output from ifconfig looks like this:
Code:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:00:00:00:00:00
inet 192.168.0.10 netmask 0xffffff00 broadcast 192.168.0.255
inet 192.168.0.11 netmask 0xffffffff broadcast 192.168.0.11
inet 192.168.0.12 netmask 0xffffffff broadcast 192.168.0.12
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
Output from "netstat -r" is following:
Code:
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.0.1 UGS em0
localhost link#3 UH lo0
192.168.0.0/24 link#1 U em0
192.168.0.10 link#1 UHS lo0
192.168.0.11 link#1 UHS lo0
192.168.0.11/32 link#1 U em0
192.168.0.12 link#1 UHS lo0
192.168.0.12/32 link#1 U em0
Do I interpret netstat correctly, if I'll say that skipping of a block happens due to jails ips magically going through lo0, on which there is a skip?
The second question is, how to set two jails on the host so that, one may talk to another over one port, with everything else blocked?
It seems to me that most common examples are such that pf cannot block unauthorized crosstalk between jails. Please, show me example otherwise.
Put on your adversarial mood, imagine that you busted process in a jail, cannot break out of the jail, yet, try to connect to neighboring jail. If you can connect, it is the issue I am talking about, if you cannot connect, can you share your network, pf and jail settings.
I tried to put jails on lo1, but had to add skip on it as well, which allows authorized cross-talk. Experiments with giving jails ips on tap's, also added ip to lo0 relationship in netstat.
What am I missing here?
Thank you, and ... he-e-elp!
PS all of this has been tested on FreeBSD 11.0.
[UPDATE] in this thread below we have one solution that involves keeping every single jail on its own loopback, having filtering on loopbacks and skip on host's lo0. It is possible to filter inter-jails communication in this setting. The caveate is that jail cannot have external ip attached to it. So, to serve traffic from outside, port(s) need to be pf-ed inside.
Hope, this will help someone.
Do leave comments, suggestions, questions.
Last edited: