ZFS on iSCSI shares

B

badtux

Due to accident of history I have several large iSCSI storage arrays. So of course the only thing to do is make a ZFS pool RAIDZ'ed across them. So I created several iSCSI shares on the arrays and gave permissions for my FreeBSD box to connect to them, created an /etc/iscsi.conf with the info on how to connect to them, issued the appropriate iscontrol commands to connect, added the resulting block devices to a ZFS RAIDZ pool, voila.

Here's the problem: On a reboot, how do I get my iSCSI initiators connected before ZFS comes up and tries to mount its pools? Thus far, the only way I can figure to do this is to hack the /etc/rc.d/zfs script at the very top of zfs_start to start up the initiators in via iscontrol (and pausing for five seconds or so to let the connects happen) before falling into the start routine proper. Otherwise my initiators don't start up before ZFS, ZFS doesn't find its pool, and ZFS says "eh, I'm just quitting" and gives up.

Am I just fundamentally misunderstanding something here, or is there really no way to get those iSCSI initiators auto-starting (other than via this hack) before ZFS tries to mount the pools on them?
 
man rcorder(8).

Inject your own iscontrol rc script into /etc/rc.d with
Code: 
REQUIRE: mountcritlocal
PROVIDE: iscsi
BEFORE: zfs
 
Last edited by a moderator:
Thanks

I did get it working, but eventually had to abandon the FreeBSD platform because the FreeBSD iSCSI initiator doesn't support immediate data mode on writes and thus was too slow when writing data to support my ZFS application (a backup system with redundancy and snapshotting). Unfortunately while I know what the problem is and how to fix it, I can't fix it because I work for a vendor of proprietary iSCSI systems that has their own proprietary iSCSI initiator and thus that whole legal thing intervenes (even though I have nothing to do with my employer's iSCSI initiator). I ended up using ZFS on another platform that had a fully functional iSCSI initiator. A bummer, really, because immediate data mode and ZFS are a great combination since ZFS is guaranteed consistent while a failure of an iSCSI subsystem with any other filesystem while using immediate data mode will result in potentially the entire filesystem becoming unusable. But it appears nobody else in the FreeBSD world ever thought of combining iSCSI and ZFS in this way...
 
IET is a target, not an initiator.

I browsed the ports collection looking for another initiator but all I found was targets. Targets don't connect to other iSCSI storage, they export local storage as iSCSI storage.
 
I wish. The FreeBSD iSCSI initiator is really weak and lacks a lot of the functionality of the Linux OpeniSCSI initiator, and performs *much* worse for reasons I mentioned above. Unfortunately as I noted above I am not in a position to do anything about that situation due to legal reasons (grrrr!).
 
Last edited by a moderator:
badtux said:
I did get it working, but eventually had to abandon the FreeBSD platform because the FreeBSD iSCSI initiator doesn't support immediate data mode on writes and thus was too slow when writing data to support my ZFS application (a backup system with redundancy and snapshotting).
Click to expand...

Can you share what OS you are picking finally?
Wish FreeBSD has better support for iSCSI for both initiator and target like NFS.
 
So FreeBSD 10+ has support for immediate mode (both iscsid(8) and ctld(8) so initiator and target). It's in the code but not in the mans so tread carefully, but it's ON by default on both types.

Going to test it out :) Can't wait for zfs mirror over zfs shares.
 
You must log in or register to reply here.
Back
Top