Identical Jails set-up fails

chigurh · Jun 30, 2012

Hi,

I have a special requirement for using two jails (NO_OPENSSL) and sysutils/ezjail doesn't offer what I need. I had world build in /usr/src and I executed the following -

[CMD=]mkdir /home/apogee/jails/cfengine /home/apogee/jails/cfengine/dev /home/apogee/jails/cfengine/etc /home/apogee/jails/cfengine/tmp[/CMD]
[CMD=]chmod 777 /home/apogee/jails/cfengine/tmp[/CMD]

[CMD=]make installworld DESTDIR=/home/apogee/jails/cfengine[/CMD]
[CMD=]make distribution DESTDIR=/home/apogee/jails/cfengine NO_OPENSSL=YES[/CMD]

[CMD=]cd /home/apogee/jails/cfengine[/CMD]
[CMD=]mount -t devfs devfs /home/apogee/jails/cfengine/dev[/CMD]
[CMD=]mount -t procfs procfs /home/apogee/jails/cfengine/dev[/CMD]
[CMD=]ln -s dev/null kernel[/CMD]
[CMD=]touch /home/apogee/jails/cfengine/etc/fstab[/CMD]
[CMD=]cp /etc/resolv.conf /home/apogee/jails/cfengine/etc/resolv.conf [/CMD]

I have replicated this setup for /home/apogee/jails/cfengine-master with appropriate path changes at various stages.

My /etc/rc.conf jail section -

Code:

jail_enable="YES"
jail_list="cfengine cfengine-master" 
jail_cfengine_rootdir="/home/apogee/jails/cfengine"
jail_cfengine_hostname="cf.BSD.biz"
jail_cfengine_ip="10.0.0.20"
jail_cfengine_devfs_enable="YES" 
jail_cfengine_devfs_ruleset="system"
jail_cfengine_procfs_enable="YES"
jail_cfengine_exec_start="/bin/sh /etc/rc"
jail_cfengine_stop="/bin/sh /etc/rc.shutdown"
jail_cfengine-master_rootdir="/home/apogee/jails/cfengine-master"
jail_cfengine-master_hostname="cfm.BSD.biz"
jail_cfengine-master_ip="10.0.0.21"
jail_cfengine-master_devfs_enable="YES"
jail_cfengine-master_devfs_ruleset="system"
jail_cfengine-master_procfs_enable="YES"
jail_cfengine-master_exec_start="/bin/sh /etc/rc"
jail_cfengine-master_stop="/bin/sh /etc/rc.shutdown"

Contents from /etc/devfs.rules

Code:

[system=10]
add hide
add path null unhide
add path zero unhide
add path random unhide
add path urandom unhide
add path crypto unhide
add path stdin unhide
add path stdout unhide
add path stderr unhide

My networking setup in /etc/rc.conf

Code:

ifconfig_re0="inet 10.0.0.5 netmask 255.255.255.0"
ifconfig_re0_alias0="inet 10.0.0.20 netmask 255.255.255.255"
ifconfig_re0_alias1="inet 10.0.0.21 netmask 255.255.255.255"
ifconfig_vlan0="inet 192.168.1.1 netmask 255.255.255.0 vlan 5 vlandev re0"

My rc.conf in jails -
/home/apogee/jails/cfengine/rc.conf

Code:

hostname="cf.BSD.biz"
network_interfaces="re0"
kern_securelevel_enable="YES"
kern_securelevel="1"
rpcbind_enable="NO"
cron_flags="$cron_flags -J 15"
syslogd_flags="-ss"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
sshd_enable="YES"
clear_tmp_enable="YES"

/home/apogee/jails/cfengine-master/rc.conf

Code:

hostname="cfm.BSD.biz"
network_interfaces="re0"
kern_securelevel_enable="YES"
kern_securelevel="1"
rpcbind_enable="NO"
cron_flags="$cron_flags -J 15"
syslogd_flags="-ss"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
sshd_enable="YES"
clear_tmp_enable="YES"

These rc.conf files actually sync and carry the same hostname. Here is the console ouput -

Code:

apogee# nano /home/apogee/jails/cfengine-master/etc/rc.conf
  GNU nano 2.2.6                             File: /home/apogee/jails/cfengine-masteretc/rc.conf

hostname="cfm.BSD.biz"
network_interfaces="re0"
kern_securelevel_enable="YES"
kern_securelevel="1"
rpcbind_enable="NO"
cron_flags="$cron_flags -J 15"
syslogd_flags="-ss"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
sshd_enable="YES"
clear_tmp_enable="YES"
                                                      [ Wrote 14 lines ]

apogee# cat /home/apogee/jails/cfengine/etc/rc.conf
hostname="cfm.BSD.biz"
network_interfaces="re0"
kern_securelevel_enable="YES"
kern_securelevel="1"
rpcbind_enable="NO"
cron_flags="$cron_flags -J 15"
syslogd_flags="-ss"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
sshd_enable="YES"
clear_tmp_enable="YES"

I don't understand why it syncs the files and carries same hostname. When I start jails with [CMD=]/etc/rc.d/jail start cfengine[/CMD] I get this -

Code:

/etc/rc.conf: jail_cfengine-master_rootdir=/home/apogee/jails/cfengine-master: not found
jail_cfengine-master_hostname=cfm.BSD.biz: not found
jail_cfengine-master_ip=10.0.0.21: not found
jail_cfengine-master_devfs_enable=YES: not found
jail_cfengine-master_devfs_ruleset=system: not found
jail_cfengine-master_procfs_enable=YES: not found
/etc/rc.conf: jail_cfengine-master_exec_start=/bin/sh /etc/rc: not found
/etc/rc.conf: jail_cfengine-master_stop=/bin/sh /etc/rc.shutdown: not found
Configuring jails:.
Starting jails: cf.BSD.biz.

[CMD=]jls[/CMD] shows cfengine jail active.

Code:

   JID  IP Address      Hostname                      Path
     2  10.0.0.20       cf.BSD.biz                    /home/apogee/jails/cfengine

It throws these errors for cfengine-master but the directory is there -
[CMD=]ls -l /home/apogee/jails/cfengine-master[/CMD]

Code:

total 46
-rw-r--r--   2 root  wheel  1012 Jun 30 06:47 .cshrc
-rw-r--r--   2 root  wheel   261 Jun 30 06:47 .profile
-r--r--r--   1 root  wheel  6200 Jun 30 06:47 COPYRIGHT
drwxr-xr-x   2 root  wheel  1024 Jun 30 06:46 bin/
drwxr-xr-x   7 root  wheel   512 Jun 30 06:47 boot/
dr-xr-xr-x   2 root  wheel   512 Jun 30 06:45 dev/
drwxr-xr-x  20 root  wheel  2048 Jun 30 06:49 etc/
lrwxr-xr-x   1 root  wheel     8 Jun 30 06:48 kernel@ -> dev/null
drwxr-xr-x   3 root  wheel  1536 Jun 30 06:46 lib/
drwxr-xr-x   2 root  wheel   512 Jun 30 06:47 libexec/
drwxr-xr-x   2 root  wheel   512 Jun 30 06:46 media/
drwxr-xr-x   2 root  wheel   512 Jun 30 06:46 mnt/
dr-xr-xr-x   2 root  wheel   512 Jun 30 06:46 proc/
drwxr-xr-x   2 root  wheel  2560 Jun 30 06:46 rescue/
drwxr-xr-x   2 root  wheel   512 Jun 30 06:47 root/
drwxr-xr-x   2 root  wheel  2560 Jun 30 06:47 sbin/
lrwxr-xr-x   1 root  wheel    11 Jun 30 06:46 sys@ -> usr/src/sys
drwxrwxrwt   2 root  wheel   512 Jun 30 06:45 tmp/
drwxr-xr-x  14 root  wheel   512 Jun 30 06:47 usr/
drwxr-xr-x  22 root  wheel   512 Jun 30 06:46 var/

Log for cfengine

Code:

/etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
Generating host.conf.
Creating and/or trimming log files.
ln: /dev/log: Operation not permitted
Starting syslogd.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Clearing /tmp (X related).
Updating motd:.
Starting cron.

Sat Jun 30 14:02:49 UTC 2012
/var/log/jail_cfengine_console.log (END)

When I run [CMD=]/etc/rc.d/jail start cfengine-master[/CMD] the process goes idle -

Code:

/etc/rc.conf: jail_cfengine-master_rootdir=/home/apogee/jails/cfengine-master: not found
jail_cfengine-master_hostname=cfm.BSD.biz: not found
jail_cfengine-master_ip=10.0.0.21: not found
jail_cfengine-master_devfs_enable=YES: not found
jail_cfengine-master_devfs_ruleset=system: not found
jail_cfengine-master_procfs_enable=YES: not found
/etc/rc.conf: jail_cfengine-master_exec_start=/bin/sh /etc/rc: not found
/etc/rc.conf: jail_cfengine-master_stop=/bin/sh /etc/rc.shutdown: not found
Configuring jails:.
Starting jails:
load: 1.91  cmd: sh 42151 [running] 8.99r 1.82u 4.58s 41% 3768k
load: 1.19  cmd: sh 42151 [running] 2164.50r 1498.63u 577.34s 100% 51656k
load: 1.19  cmd: sh 42151 [running] 2881.21r 2027.66u 750.14s 100% 59876k
load: 1.26  cmd: sh 42151 [running] 2883.85r 2029.62u 750.77s 100% 59900k
load: 1.26  cmd: sh 42151 [running] 2884.07r 2029.79u 750.81s 100% 59908k
load: 1.26  cmd: sh 42151 [running] 2884.33r 2029.99u 750.87s 100% 59908k
load: 1.26  cmd: sh 42151 [running] 2884.56r 2030.17u 750.92s 98% 59912k
load: 1.26  cmd: sh 42151 [running] 2884.76r 2030.33u 750.95s 100% 59912k
load: 1.26  cmd: sh 42151 [running] 2884.95r 2030.48u 750.99s 100% 59912k
load: 1.26  cmd: sh 42151 [runnable] 2885.14r 2030.65u 751.01s 100% 59912k
load: 1.26  cmd: sh 42151 [running] 2885.33r 2030.78u 751.06s 100% 59920k
load: 1.26  cmd: sh 42151 [running] 2887.44r 2032.48u 751.44s 100% 59936k
load: 1.24  cmd: sh 42151 [running] 2887.92r 2032.86u 751.54s 100% 59944k
load: 1.35  cmd: sh 42151 [running] 3948.27r 2827.40u 998.06s 100% 70272k
load: 1.30  cmd: sh 42151 [running] 3959.68r 2836.29u 1000.40s 100% 70364k

I am sure I have messed up while configuring network, this is my first attempt with FreeBSD jails. Please help me fix this.

Best,

Chigurh
[CMD=]uname -a[/CMD]

Code:

FreeBSD apogee.BSD.biz 8.3-STABLE FreeBSD 8.3-STABLE #0: Sat Jun 23 05:02:43 PDT 2012
apogee@apogee.BSD.biz:/usr/obj/usr/src/sys/INFERNO  amd64

SirDice · Jul 9, 2012

You can't use "-" in a variable's name.

da1 · Jul 11, 2012

Maybe try ezjail ?

chigurh · Jul 14, 2012

rc.conf does not read '-'. After changing cfengine-master to cfenginemaster I was able to configure jails.

Identical Jails set-up fails

chigurh

SirDice

Administrator

da1

chigurh