Solved FreeBSD Jail network issues

S

SeppyBSD

Hallo,

I have a network issue at my jail system:

I used following Tutorial:
https://www.davd.eu/posts-freebsd-jails-with-a-single-public-ip-address/

Now I made ezjail-admin console ftp and tried to make pkg update but it did not work.

my /etc/rc.conf
Code: 
hostname="jail"
keymap="german.iso.kbd"
ifconfig_vtnet0="DHCP"
ifconfig_vtnet0_ipv6="inet6 accept_rtadv"
sshd_enable="YES"
moused_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
ezjail_enable="yes"
pf_enable="yes"
cloned_interfaces="lo1"
ipv4_addrs_lo1="192.168.0.1-9/29"

my /etc/pf.conf
Code: 
#       $FreeBSD: releng/11.1/share/examples/pf/pf.conf 293862 2016-01-14 01:32:17Z kevlo $
#       $OpenBSD: pf.conf,v 1.34 2007/02/24 19:30:59 millert Exp $
#
# See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
# Remember to set gateway_enable="YES" and/or ipv6_gateway_enable="YES"
# in /etc/rc.conf if packets are to be forwarded between interfaces.

#ext_if="ext0"
#int_if="int0"

#table <spamd-white> persist

#set skip on lo

#scrub in

#nat-anchor "ftp-proxy/*"
#rdr-anchor "ftp-proxy/*"
#nat on $ext_if inet from !($ext_if) -> ($ext_if:0)
#rdr pass on $int_if proto tcp to port ftp -> 127.0.0.1 port 8021
#no rdr on $ext_if proto tcp from <spamd-white> to any port smtp
#rdr pass on $ext_if proto tcp from any to any port smtp \
#       -> 127.0.0.1 port spamd

#anchor "ftp-proxy/*"
#block in
#pass out

#pass quick on $int_if no state
#antispoof quick for { lo $int_if }

#pass in on $ext_if proto tcp to ($ext_if) port ssh
#pass in log on $ext_if proto tcp to ($ext_if) port smtp
#pass out log on $ext_if proto tcp from ($ext_if) to port smtp
#pass in on $ext_if inet proto icmp from any to ($ext_if) icmp-type { unreach, redir, timex }
PORT_WWW="80"
IP_PUB="192.168.124.59"
IP_JAIL="192.168.0.2"
NET_JAIL="192.168.0.0/24"
PORT_JAIL="{80,443,2020}"
scrub in all
nat pass on re0 from $NET_JAIL to any -> $IP_PUB
rdr pass on re0 proto tcp from any to $IP_PUB port $PORT_WWW -> $IP_JAIL

What am I doing wrong?

Thanks for answer
 
You haven't enabled routing, add to /etc/rc.conf:
Code: 
gateway_enable="YES"
 
still no luck. pkg update does not work. the output of ifconfig:

Code: 
$ ifconfig
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
   ether 52:54:00:32:91:5d
   hwaddr 52:54:00:32:91:5d
   inet6 fe80::5054:ff:fe32:915d%vtnet0 prefixlen 64 scopeid 0x1
   inet 192.168.124.59 netmask 0xffffff00 broadcast 192.168.124.255
   nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
   media: Ethernet 10Gbase-T <full-duplex>
   status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
   options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
   inet6 ::1 prefixlen 128
   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
   inet 127.0.0.1 netmask 0xff000000
   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
   groups: lo
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
   options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
   inet 192.168.0.1 netmask 0xfffffff8
   inet 192.168.0.2 netmask 0xffffffff
   inet 192.168.0.3 netmask 0xffffffff
   inet 192.168.0.4 netmask 0xffffffff
   inet 192.168.0.5 netmask 0xffffffff
   inet 192.168.0.6 netmask 0xffffffff
   inet 192.168.0.7 netmask 0xffffffff
   inet 192.168.0.8 netmask 0xffffffff
   inet 192.168.0.9 netmask 0xffffffff
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
   groups: lo
 
"Does not work" is pretty ambiguous. What errors are you getting?
 
following message:
Code: 
root@ftpserv:~ # pkg update
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly, please wait...
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly/Latest/pkg.txz: No address record
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.
root@ftpserv:~ #

and the ifconfig of the jail:
Code: 
root@ftpserv:~ # ifconfig
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
   ether 52:54:00:32:91:5d
   hwaddr 52:54:00:32:91:5d
   media: Ethernet 10Gbase-T <full-duplex>
   status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
   options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
   groups: lo
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
   options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
   inet 192.168.0.2 netmask 0xffffffff
   groups: lo
root@ftpserv:~ #
 
"No address record" means your DNS resolving isn't working. Check /etc/resolv.conf in the jail.
 
The resolv inside the jail:
Code: 
# Generated by resolvconf
nameserver 192.168.124.1

The resolv.conf of host:
Code: 
# Generated by resolvconf
nameserver 192.168.124.1
 
Ok, that looks good. This probably means the jail doesn't have network or your NAT on the host isn't set up correctly. The easiest to troubleshoot this is to enable raw sockets on the jail temporarily. That will allow you to ping from within the jail (this is normally not allowed). Ping something outside your network and check with tcpdump(8) on the host's external interface to see if your packets are correct.
 
Good catch forquare

I would suggest changing the pf.conf and make use of the variables.

Code: 
ext_if="vtnet0"

NET_JAIL="192.168.0.0/24"

nat on $ext_if from $NET_JAIL to any -> ($ext_if)
 
SirDice said:
Good catch forquare

I would suggest changing the pf.conf and make use of the variables.

Code: 
ext_if="vtnet0"

NET_JAIL="192.168.0.0/24"

nat on $ext_if from $NET_JAIL to any -> ($ext_if)
Click to expand...

nat on $ext_if ....

pf says syntax error.

even "nat pass on "...

both are syntax errors (pf says)

EDIT:

I can't ping even after:
"sysctl security.jail.allow_raw_sockets=1" on host
 
Nothing wrong with the syntax. You probably forgot to define the variable.

Code: 
dice@maelcum:~/test % pfctl -nf pf.conf
dice@maelcum:~/test % cat pf.conf
ext_if="vtnet0"

NET_JAIL="192.168.0.0/24"

nat on $ext_if from $NET_JAIL to any -> ($ext_if)
 
SirDice said:
Nothing wrong with the syntax. You probably forgot to define the variable.

Code: 
dice@maelcum:~/test % pfctl -nf pf.conf
dice@maelcum:~/test % cat pf.conf
ext_if="vtnet0"

NET_JAIL="192.168.0.0/24"

nat on $ext_if from $NET_JAIL to any -> ($ext_if)
Click to expand...

Ok I found the mistake. Your code is correct but did not make the jail getting network.

after ezjail-admin create jailname 192.168.0.2 I get this Warning:
Code: 
Warning: Some services already seem to be listening on all IP, (including 192.168.0.2)
  This may cause some confusion, here they are:
root     syslogd    534   6  udp6   *:514                 *:*
root     syslogd    534   7  udp4   *:514                 *:*
 
Make sure all services on the host are bound to the host's IP address. And I would suggest removing this:
Code: 
ipv4_addrs_lo1="192.168.0.1-9/29"
The IP addresses will get automatically created/removed when you start/stop the jail, there's no need to create them beforehand.

As for raw sockets, you need to modify the EZJail configuration in /usr/local/etc/ezjail/.

Code: 
export jail_<jailname>_parameters="allow.raw_sockets=1"
 
SirDice said:
Make sure all services on the host are bound to the host's IP address. And I would suggest removing this:
Code: 
ipv4_addrs_lo1="192.168.0.1-9/29"
The IP addresses will get automatically created/removed when you start/stop the jail, there's no need to create them beforehand.

As for raw sockets, you need to modify the EZJail configuration in /usr/local/etc/ezjail/.

Code: 
export jail_<jailname>_parameters="allow.raw_sockets=1"
Click to expand...

aren't the services bound to the host's IP automatically? if not how do I bind them?
 
aren't the services bound to the host's IP automatically? if not how do I bind them?
Click to expand...
By default most services automatically bind to all IP addresses on the host, this would include all the IP addresses you assigned to jails.

How you do this would depend on the service. Some need additional flags in rc.conf while others need to have their configuration files modified.
 
Finished.

The Error was the create command.

I deleted Jail and created one via:

ezjail-admin create ftpexample 'lo1|127.0.1.1,vtnet0|192.168.0.2'


worked fine. Now PKG update works without problems
 
You must log in or register to reply here.
Back
Top