Jails setup with the external IP address on vtnet0

[r39]

Hello,

I'm trying to setup jails on a Digital Ocean droplet by following the instructions in the handbook: https://www.freebsd.org/doc/handbook/jails-ezjail.html
The network configuration is like this:

* vtnet0 with the external IP address, 10.10.0.5 netmask 0xffff0000 (Digital Ocean's internal network), and inet 192.168.1.50 netmask 0xffffffff (the jail; per the instructions in the handbook)
* lo0 127.0.0.1 netmask 0xff000000
* lo1 127.0.1.1 netmask 0xffffff00 (the jail)

I copied the /etc/resolv.conf from the host system to the jail. When using console on the jail, I try pkg update and get the following error:

root@webserver:~ # pkg update
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly, please wait...
\pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly/Latest/pkg.txz: No address record

After reading this article, I also tried to use a private network IP address for the jail and set up NAT using ipfw like this:

#!/bin/sh

ipfw -q -f flush       # Delete all rules

add="ipfw -q add "     # build rule prefix
nat="ipfw -q nat "

ipfw disable one_pass

# Loopback interface
$add 100 allow ip from any to any via lo0

# NAT inbound packets
$nat 1 config if vtnet0 unreg_only reset \
    redirect_port tcp 172.16.1.1:443 443 \
    redirect_port tcp 172.16.1.1:80 80
$add 700 reass all from any to any in
$add 800 nat 1 ip from any to any in via vtnet0

# Stateful rules
$add 900 check-state

# Allow any connection out
$add 1000 allow tcp from me to any out established
$add 1100 allow tcp from me to any out setup keep-state
$add 1200 allow udp from me to any out keep-state
$add 1300 allow icmp from me to any out keep-state
$add 1400 allow ipv6-icmp from me to any out keep-state

# For pinging the IP address
$add 1900 allow icmp from any to me in icmptypes 8
$add 2000 allow ipv6-icmp from any to me in icmp6types 128,129

# Deny fragments
$add 2300 deny all from any to any frag in via vtnet0

# ssh
$add 2400 allow tcp from [redacted] to me 22 in via vtnet0

# http/https
$add 2500 allow tcp from any to me 80 in via vtnet0 setup limit src-addr 10
$add 2600 allow tcp from any to me 443 in via vtnet0 setup limit src-addr 10

# NAT jails
$add 2700 skipto 10000 tcp from 172.16.0.0/12 25,37,53,80,443 to any out via vtnet0 setup keep-state
$add 2800 skipto 10000 udp from 172.16.0.0/12 37,53 to any out via vtnet0 keep-state

$add 9000 count ip from any to any

# Deny by default
$add 9999 deny log logamount 500 ip from any to any

# NAT jails
$add 10000 nat 1 ip from any to any out via vtnet0
$add 10100 allow ip from any to any

All in vain... It still does not work. So I have two questions:

1) How do I set up jails if the external IP address is on vtnet0? The machine has no real network card, only vtnet0. Do I need to use a private network IP address and NAT in this case?

2) How do I test network connection from inside the jail? curl/telnet is not installed, ping is prohibited. How can I test that I can reach a particular IP address in these conditions?

Thank you for your help.

 

[sadaszewski]

I use PF so can't say anything about IPFW. But this normally works:

Edit /etc/pf.conf:

ext_if="vtnet0"
jail_ip="127.0.1.1"
nat pass on $ext_if from $jail_ip to any -> ($ext_if)
rdr pass on $ext_if proto tcp from any to ($ext_if) port {443, 80} -> $jail_ip

Call:
sysrc pf_enable=YES
service pf start

BTW. Why not try out https://github.com/sadaszewski/focker/ it is much easier and more powerful than ezjail.

 

[fbsd1]

I use qjail and it works. Good documentation. ezjail is obsolete and no longer supported since FBSD 9.0. Do you want your non-vnet jails to have public internet access? qjail create -4 192.168.1.50 jailname then all you have to do is NAT 192.168.0.0/16. Your on your own when it comes to ipfw because I use ipfilter firewall.

 

[Lamia]

You may want to keep an eye on this thread - https://forums.freebsd.org/threads/jail-network-interface-lo.76054/#post-468396 - too. They're trying to achieve the same thing. The firewall implementations may vary though. The tips on nat/pf work and can be revised for ipfilter/ipfw.

 

[r39]

I use qjail and it works.

Thank you. As far as I know, qjail is an unauthorized fork of ezjail (https://erdgeist.org/posts/2017/dont-piss-in-my-beer.html) and the latest version was released 3 years ago. But if there is nothing better, I would use it.

Ezjail is a nice shell script, with no dependencies. I wonder why it's not supported anymore, while being documented in the FreeBSD handbook.