Jails, ezjail, CARP, FreeBSD10

[pathiaki]

Hi,

Well, I just saw that FreeBSD 10 did a full rewrite on CARP. Nice. However, I have a major problem. There's not a lot out there.

I have floating storage that has all my jails on it. The ezjail software in ports is managing it. Now, I believe I have the configuration correct for ifconfig in the /etc/rc.conf on the master and slave, respectively:

ifconfig_ix0="inet 10.2.0.51 netmask 255.255.255.248.0"
ifconfig_ix0_alias0="inet vhid 1 pass <mypassword> alias 10.2.0.53/32"

ifconfig_ix0="inet 10.2.0.51 netmask 255.255.255.248.0"
ifconfig_ix0_alias0="inet vhid 1 advskew 100 pass <mypassword> alias 10.2.0.53/32"

In the file /etc/devd.conf on both machines, I have:

notify 0 {
     match "system"       "CARP"
     match "subsystem"  "[0-9+@[0-9a-z]+";
     match "type"          "(MASTER|BACKUP)"'
     action                   "/usr/local/bin/carpmaster.sh"

The permissions/ownership on the action file are root:wheel and 774.

When I fail one machine by shutting it down, the floating storage (It's fiber channel connected to both machines via identical cards.) doesn't mount.

The main code in /usr/local/bin/carpmaster.sh, outside of logging directives, and other things, performs this:

zpool import -f mypool 
/usr/local/etc/rc.d/ezjail restart

There is no issue between CARP detection and BACKUP changing to MASTER on either machine in either direction.

The problem is the mounting of the storage and restarting of the jails.

Neither seems to occur and I'm at a loss. The logging states that everything is occurring but things just aren't showing up.

Pointers?

Thank you,

P.

 

[Beeblebrox]

zpool import -f mypool
/usr/local/etc/rc.d/ezjail restart

I'm going to take a leap and guess that the zpool import part is a fail-safe (notice the -f{orce} flag) to get the jail working. This means that ZFS is most likely not an issue.

Try restarting the ezjails directly, without passing through the CARP script ( ezjail restart). Do all your jails start? If not, check the relevant /etc/fstab.<jail_name> file. You might have to temporarily change the network settings in the /usr/local/etc/ezjail/<jail_name> file while checking whether the jail gets started. Also, jls(8) of course!

I have the ezjail framework, but use /etc/jail.conf for setup. Create the needed /etc/fstab (very simple really) then edit /etc/jail.conf as needed and set in /etc/rc.conf

jail_list="jail1 jail2"

This way, any conflicting setting in the ezjail configuration files or ezjail script gets by-passed/ignored. The native jail script takes over, uses your rc.conf settings and mounts via fstab. My hunch is you have conflicting network settings and you need to let the jail.conf handle that rather than rc.conf. Example rc.conf entry (modify as required by CARP):

jail1 {
        name = jail1;
        path = /data/ezjail/jail1;
        interface = lo2;
        ip4.addr = 192.168.2.98/32;
        host.hostname = my.jail;
        devfs_ruleset = xx;
        mount.devfs;
        mount.fstab = /etc/fstab.jail1;
}

Ezjail is still great for the simple update and basejail mechanism.

(It's fiber channel connected to both machines

Oh yes, baby...

 

[storvi_net]

Hey,

Can you give some additional information about your storage(configuration)? I am looking for a HA-solution and try to find some experiences in combination with CARP / HAST / FC, etc.

Regards
Markus