FreeBSD update 9.2->10 ; ezjail config

_martin

_martin

Hi,

Yesterday I upgraded my server from 9.2 to 10.0 using sources. Pretty much business as usual. Everything seemed to be working fine. I did notice a warning when my ezjail jails were starting:

Code: 
Feb 11 01:43:59 foxi root: /etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider to migrate to /etc/jail.conf.
It does sound like a warm warning. Jails did start, services inside were working as expected. But I did notice something - I was able to control PF within the jail (/dev/pf exists in each jail; here webserver is a jail):

Code: 
root@webserver:/root # pfctl -d
pf disabled
root@webserver:/root #

Uh, oh. What ?
Another example: /dev/ada0 is one disk in rpool:

Code: 
root@webserver:/root # hd -n 64 /dev/ada0p1
00000000  31 c9 8e c1 8e d9 8e d1  bc 00 7c bb 63 7c 8b 77  |1.........|.c|.w|
00000010  0a 01 de 89 f0 c1 e8 04  83 e6 0f 8e d8 83 c6 f0  |................|
00000020  b8 00 0a bf f0 ff 8e c0  fd 89 f9 41 f3 a4 8e d9  |...........A....|
00000030  8e c1 8b 4f 0a 89 de bf  00 90 01 ce 01 cf 4e 4f  |...O..........NO|
00000040
root@webserver:/root #
New 10.0 /etc/defaults/devfs.rules is installed, nothing was changed prior to upgrade (same ezjail configs).

I do believe that problem is somewhere between keyboard and chair, but this is very unpleasant issue considering the warning I received. I'm rechecking handbook to see what I did wrong, posting here as FYI - maybe somebody else didn't notice this either.
 
matoatlantis said:
yesterday I upgraded my server from 9.2 to 10.0 using sources. Pretty much business as usual. Everything seemed to be working fine. I did notice a warning when my ezjail jails were starting:

Code: 
Feb 11 01:43:59 foxi root: /etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider to migrate to /etc/jail.conf.
It does sound like a warm warning. Jails did start, services inside were working as expected.
Click to expand...

This is just letting you know that you should migrate to using /etc/jail.conf to configure your jails instead of rc variables. If I remember right, if you look in /var/run there will be generated files showing you the snippet used for each of the jails.

But I did notice something - I was able to control PF within the jail (/dev/pf exists in each jail; here webserver is a jail):

Code: 
root@webserver:/root # pfctl -d
pf disabled
root@webserver:/root #

Uh, oh. What ?
Another example: /dev/ada0 is one disk in rpool:

Code: 
root@webserver:/root # hd -n 64 /dev/ada0p1
00000000  31 c9 8e c1 8e d9 8e d1  bc 00 7c bb 63 7c 8b 77  |1.........|.c|.w|
00000010  0a 01 de 89 f0 c1 e8 04  83 e6 0f 8e d8 83 c6 f0  |................|
00000020  b8 00 0a bf f0 ff 8e c0  fd 89 f9 41 f3 a4 8e d9  |...........A....|
00000030  8e c1 8b 4f 0a 89 de bf  00 90 01 ce 01 cf 4e 4f  |...O..........NO|
00000040
root@webserver:/root #
New 10.0 /etc/defaults/devfs.rules is installed, nothing was changed prior to upgrade (same ezjail configs).

I do believe that problem is somewhere between keyboard and chair, but this is very unpleasant issue considering the warning I received. I'm rechecking handbook to see what I did wrong, posting here as FYI - maybe somebody else didn't notice this either.
Click to expand...

You probably need to put devfs_load_rulesets="YES" in your rc.conf to get the rulesets to work.

Also, one thing to be aware of if you're using ezjail (or freebsd-update -b for that matter) is that it doesn't properly update your jails (http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/186152).
 
Thanks for the tip about new jail configs in /var/run and an update problem. I was actually updating it from sources, so I didn't hit that one.
Right though, warning is harmless and just informs that new config should be used (for which new ezjail script has to be released).

As everything was working just fine before OS update - jails were not able to see /dev/pf before and /etc/rc.conf is the same as it was on 9.2, I thought problem is with the new jail config location. I didn't have devfs_load_rulesets="YES" on 9.2 and it was working fine.
On 10 it seems this toggle has to be in rc.conf.

Thanks for your help.
 
On FreeBSD 10 devfs rules must be integers rather than names. You'll have to update your jail configuration files to use numbers instead.

This is what I did, I have some additional rules so I just updated all my files with a sed one-liner.
sed -i '' -e 's/"devfsrules_jail"/"4"/;s/"devfsrules_jail_with_vpn"/"5"/;s/"devfsrules_jail_with_bpf"/"6"/' /usr/local/etc/ezjail/*
 
Nope, nothing helps.

With FBSD10 FreeBSD 10 fresh install with sysutils/ezjail, I had to put
Code: 
devfs_load_rulesets="YES"
in /etc/rc.conf to hide devices from jails. And then restart devd and jails. It works with numbers and/or names. It would be quite helpful if somebody considered notifying users about a change like this.

Hopefully sysutils/ezjail port maintainers will adapt ezjail to cope with /etc/jail.conf and the devfs surprise ;)
 
@junovitch That doesn't seem to be the problem; it's not working if you just switch from names to numbers. It seems devfs_load_rulesets="YES" is a must in /etc/rc.conf.

I tried to start jail "manually" from command line using jail on fresh FreeBSD 10 installation without that toggle in /etc/rc.conf and the same issue occurred. I have yet to find what was changed that it is now needed.

I'd consider this behavior as a bug. Following command was used:

Code: 
jail -c path=/local/jails/sandbox devfs_ruleset=4 mount.devfs host.hostname=sandbox ip4.addr=192.168.252.1 command=/bin/sh
There was no warning or no information of any kind that ruleset #4 was not loaded.
 
Last edited by a moderator:
You're right. Sorry I jumped the gun there because the ruleset number change was advertised in /usr/src/UPDATING entry 20131010. I came across the warnings when I upgraded and fixed all my files but I never actually looked at /dev under my jails afterwards. Now that I have I see that the rules have no effect. Based off your comments I have found it to be fixable by doing the following (note devfs and not devd was restarted, they are different):
sysrc devfs_load_rulesets=YES; service devfs restart; ezjail-admin restart
 
@junovitch Np .. Correct, that is what I did when @dpejesh mentioned the toggle. After that my jails were OK.

Question is now whether this can be considered as a bug or not. It seems to me more people hit this problem and were not aware of it. Speaking for myself: as jails started and no apparent issue occurred, I left my jails as is for a day. Some users do have root access there which is OK for jail, but for sure not for host. Root had access to firewall and disks - that's pretty bad.

I would expect jail to report that devfs rules can't be loaded as specified. Hopefully I'll have some spare time this weekend to look into it and check out the history of that toggle (why it was not needed on 9.2 but it is needed now).
Maybe then I'll open a bug report ..
 
Last edited by a moderator:
Same here

I upgraded 9.2-RELEASE -> 10-RELEASE and realized that root from jails could do for example pfctl -d out of sudden.
I would expect a warning ;)
However I don't know if it's a bug, or I just missed a warning somewhere that devfs_load_rulesets=YES in /etc/rc.conf is now required for jails to load their rulesets.
 
junovitch said:
On FreeBSD 10 devfs rules must be integers rather than names. You'll have to update your jail configuration files to use numbers instead.

This is what I did, I have some additional rules so I just updated all my files with a sed one-liner.
sed -i '' -e 's/"devfsrules_jail"/"4"/;s/"devfsrules_jail_with_vpn"/"5"/;s/"devfsrules_jail_with_bpf"/"6"/' /usr/local/etc/ezjail/*
Click to expand...

This approach just worked for me today.
 
You must log in or register to reply here.
Back
Top