Solved trouble networking a jail

[Chris_H]

I think I'm retarded today. I can't for the life of me get a freshly created jail(8) attached to any reachable IP/hostname.
Anyway, here's some detail;
I'm running 12-CURRENT as the host for the jail(8) I'm creating, so that I can build a RELENG_11 world/kernel for an old outdated box I can't take out of production long enough to build on (its) bare metal.
So I unpack all the "dists" from the (11) install media into the jail folder. Set up the basics in the hosts rc.conf(5):

# # # JAIL(8) iface # # #
cloned_interfaces="lo1"

# # # JAIL(8) stuff # # #
inetd_flags="-wW -a 127.0.0.2"
rpcbind_enable="NO"
jail_enable="YES"
jail_list="releng11"

and the hosts jail.conf(5):

allow.raw_sockets = 1;

relenf11 {
path = /jails/releng11;
host.hostname = releng11.localhost;
ip4.addr = 127.0.0.2;
interface = lo1;
exec.clean;
mount.devfs;
mount.fdescfs;
mount.procfs;
exec.start =   "/bin/sh /etc/rc";
exec.stop = "/bin/sh   /etc/rc.shutdown";
exec.consolelog = /var/log/jail.releng11.console.log;
}

copy jail section from /etc/defaults/devfs.rules to /etc/devfs.rules
exec
devfs -m /jails/releng11/dev rule -s 4 applyset
clone the lo interface:
service netif cloneup
Now I have lo1 for the jail' IP (127.0.0.2).
I copy resolv.conf(5) from the hosts /etc/ to the jail' /etc/

# the jail HOST's internet facing IP
nameserver WW.XX.YY.XYZ
# our local recursive DNS
nameserver WW.XX.YY.XYY
nameserver 127.0.0.1

Then I fire up jail(8), to initiate the jail' environment
(root password, adduser, newaliases, tzsetup)
jail -c path=/jails/releng11 command=/bin/sh
and create a rc.conf

hostname="releng11.localhost"
sshd_enable="YES"
sendmail_enable="NONE"

All goes without error.
So I fire up the (now) initiated jail:
# jail -c releng11
Still no errors. So I attempt to login:
# ssh -l root releng11
Just hangs, no connection. So I try:
# ssh -l root releng11.localhost
Nothing. So I also try the (regular) user I also created. But still
a no-go. So I check the jail' status:

# jls
   JID  IP Address      Hostname                      Path
     3  127.0.0.2       releng11.localhost              /jails/releng11

Hmm...

# pgrep -lfj 3
43901 /usr/sbin/cron -s
42643 /usr/sbin/sshd
26046 /usr/sbin/syslogd -s

All looks correct. I give up! What am I doing wrong/forgetting?
Please help.

Thanks!

--Chris

 

[Snurg]

Your jail has the IP 127.0.0.2 which cannot work, as it overlaps localhost.
Give it an ip in the 10, 172, or 192 private ranges.

I do not know for sure whether this alone is sufficient, as I always use jails in conjunction with pf, which then does more, like passing and blocking things.

 

[Chris_H]

Thanks for the reply Snurg !
I too am using pf(4). I don't suppose you'd be willing to share some hints as to how you're doing it?

Thanks again, Snurg !

--Chris

 

[Snurg]

My pf.conf is just a mess atm.
This is one of the reasons why I am going to automate it with my jail manager which is in work.
I'll extract some parts of it that are the important ones regarding jails.
But please be aware that it possibly won't work directly, as this are only snippets put together from the whole pf.conf and it is untested.
It is probably unsafe as well. The forum gurus probably will frown when they see it.
I'll work on it soon when my jail manager progressed so far that I can let it configure pf, too.

You'll have at least to change to your external interface data.

# pf.conf

# PORTS #################################################################

if_ext="xxx"               # port connected to external modem/router
net_ext="192.168.xx.xx/24"  # the (outside but private) net of the Internet router
ip_ext="192.168.xx.xx"     # the public IP (or the private after the Internet modem/router)

if_lo="{lo0 lo1 lo2 lo3 lo4 lo5 lo6 lo7 lo8 lo9 lo10 lo99 lo100 lo101}"   # loopback interfaces to jails

# JAIL NETWORKS #################################################################

net_jails="10.10.10.0/24"
ip_jail_haproxy="10.10.10.101"    # https able reverse proxy for web server jails
ip_jail_dnssec="10.10.10.99"            # DNSSEC jail

# SETTINGS #################################################################

set block-policy return
set skip on $if_lo                                    # skip checking loopback intf
scrub in on $if_ext all fragment reassemble           # clean and defragment
# REDIRECTS #################################################################

rdr pass on $if_lan proto {tcp udp} from $net_lan to $ip_lan port {domain} -> $ip_jail_dnssec
rdr pass on $if_ext proto tcp from $net_ext to $ip_ext port http -> $ip_jail_haproxy port http
rdr pass on $if_ext proto tcp from $net_ext to $ip_ext port https -> $ip_jail_haproxy port https
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"

# NAT #################################################################
nat on $if_ext from ! $if_ext to any -> ($if_ext)

block quick proto ipv6 from any to any

# PASSES #################################################################

pass                    # to establish keep state
pass all                # pass all just for seeing whether it works. Insecure!

 

[Chris_H]

Hello, Snurg , and thanks for the reply!
OK after my reply, I realized I
set skip on lo0
but NOT set skip on lo1. D'OH!
I think that was my problem. I'm going to give it another go, and see what results, and will report back shortly.

Thanks again, Snurg !

EDIT:

YEP That was it!
Simply adding lo1, as in;
set skip on { lo0, lo1 } to my pf(4) rules, allowed the jail(8) configuration I posted above to work as intended!

WooHoo!

--Chris

 

[SirDice]

I'm running 12-CURRENT as the host for the jail(8) I'm creating, so that I can build a RELENG_11 world/kernel for an old outdated box

You really don't need to run -CURRENT in order to do that. You just need to make sure the host has the same or a newer 11.1-STABLE than the jail.

 

[Chris_H]

Thanks for the reply, SirDice !
Indeed. You are quite correct. But this is also my development box for ports, and other things I maintain. So running "bleeding edge" seemed the best direction to go. It also, as you (thoughtfully) mention, insures that my ABI is new enough to build (somewhat) older versions.

Thanks again, SirDice !

--Chris