ZFS NFS sharing many ZFS datasets / home directories

W

wilschie

Hello,

I am evaluating a move from Solaris 11.4 to FreeBSD 13 and would like to know how you distribute ZFS datasets of hundreds of users (home directories) via NFS. Each user has a dataset under tank/home and there are four to six subnets for which the datasets should be made available. Under Solaris this can be done with

Code: 
zfs set share.nfs.sec.sys.rw=@172.29.2.0/24:@172.29.4.0/24 tank/home

for the two subnets 172.29.2.0/24 and 172.29.4.0/24. Each dataset under tank/home inherits the share configuration. No files have to be edited for this to work. What would be the equivalent under FreeBSD? What is documented in man zfs does not work. The example

Code: 
zfs set sharenfs='rw=@172.29.2.0/24' tank/home

gives

Code: 
Apr 21 10:02:24 bsdfs01 mountd[2547]: can't get address info for host rw=@172.29.2.0/24
Apr 21 10:02:24 bsdfs01 mountd[2547]: bad host rw=@172.29.2.0/24, skipping

and showmount -e is empty.

Searching for a solution I found that one should put the datasets one wants to export in /etc/exports. But this is very tedious and error-prone for hundreds of users, isn't it?

Thank you
Willi
 
The NFS share option is a bit of a hack on FreeBSD. It basically generates a /etc/zfs/exports that's loaded with FreeBSD's nfsd(8)/mountd(8). So you need to use FreeBSD's exports(5) format here.
 
FreeBSD's exports(5) means editing /etc/exports or /etc/zfs/exports, right!? Or do you mean using what is available in exports(5) with the set sharenfs command, like

Code: 
zfs set sharenfs='-network 172.29.2.0/24' tank/home
 
As far as I know there's some translation happening but when you add an NFS share with zfs sharenfs ... it generates an appropriate line in /etc/zfs/exports. This is different from Solaris where ZFS can talk to the NFS daemon directly through the kernel.

Due to this 'translation' I found it easier to just use /etc/exports directly. sharenfs is nice for 'simple' shares, but anything more complex is just easier to do with exports(5).
 
Code: 
zfs set sharenfs='-network=172.29.2.0/24' tank/home

works for one subnet, /etc/zfs/exports is modified

Code: 
. . .
/tank/home      -network=172.29.2.0/24
/tank/home/willi        -network=172.29.2.0/24
. . .

Mounts from clients work.

How can I add another network? Can not find how to do it in exports(5), overreading it?
 
Thank you, SirDice, for your quick help!

Unfortunately, I guess I'll have to stick with Solaris. Another issue that I had completely tuned out when playing around with FreeBSD (which has been a lot of fun and still is) is backup. We backup to an IBM Spectrum Protect infrastructure and there is no FreeBSD client for Spectrum Protect.
 
Phishfrey said:
The big hammer approach I take is to make the whole subnet available:
/tank/home -network=172.29.0.0/16
Click to expand...

There are subnets with local root users, who should not be allowed to NFS mount data shares. For such networks there is Samba. The segmentation is not completely secure, of course, but it makes it a little more difficult to get into mischief.
 
wilschie said:
for the two subnets 172.29.2.0/24 and 172.29.4.0/24. Each dataset under tank/home inherits the share configuration. No files have to be edited for this to work. What would be the equivalent under FreeBSD? What is documented in man zfs does not work.
Click to expand...
Yes, for a couple of reasons. The syntax is tricky, and mountd gives misleading warnings (bug #263011)
Nevertheless I got it working for a single subnet, so we would just need to figure out how to do it for multiple subnets:

Code: 
# zfs set sharenfs="maproot=root:wheel,mask=255.255.255.0,network=8.8.8.8" zdesk/home
# cat /etc/zfs/exports
/home   -maproot=root:wheel -mask=255.255.255.0 -network=8.8.8.8
# zfs create zdesk/home/bogus
# cat /etc/zfs/exports
/home   -maproot=root:wheel -mask=255.255.255.0 -network=8.8.8.8
/home/bogus     -maproot=root:wheel -mask=255.255.255.0 -network=8.8.8.8

That one works so far, and is nicely inherited.
The problem is with multiple subnets. This is apparently not supported for networks.exports(5):
The third component of a line specifies the host set to which the line
applies. The set may be specified in three ways. The first way is to
list the host name(s) separated by white space. (Standard Internet “dot”
addresses may be used in place of names.) The second way is to specify a
“netgroup” as defined in the netgroup file (see netgroup(5)). The third
way is to specify an Internet subnetwork using a network and network mask
that is defined as the set of all hosts with addresses within the
subnetwork.
Click to expand...
We can have multiple hosts in one line, but only one network and mask. And zfs doesn't appear to be willing to create multiple lines. So maybe that netgroup(5)(5) thing is the way to go (I never used that).

You will also need to add the -S flag to mountd_flags in /etc/rc.conf, otherwise zfs editing /etc/zfs/exports will not work smoothly.

wilschie said:
Searching for a solution I found that one should put the datasets one wants to export in /etc/exports. But this is very tedious and error-prone for hundreds of users, isn't it?
Click to expand...
That is certainly wrong. Don't do that. Have zfs maintain its exports automatically.

"Spectrum Protect" is what I know as ADSM/TSM? I'm not sure if they have ever published an api documentation, so that may be a problem indeed. Maybe they have something that runs in linux emulation?
 
Last edited:
PMc said:
"Spectrum Protect" is what I know as ADSM/TSM?
Click to expand...
Yes. IBM renamed TSM to Spectrum Protect in 2015, I think.

PMc said:
Maybe they have something that runs in linux emulation?
Click to expand...
ZFS on Linux is not supported by TSM. The datasets are backed up but the backup server tags the ZFS datasets / file systems as Filespace Type UNKNOWN. This may be a cosmetic issue, but it does not inspire confidence :)

Furthermore, under Linux, each dataset and sub dataset to be backed up must be listed in a configuration file. This is hardly feasible with hundreds of users and fluctuation among employees and colleagues. ZFS under Solaris is backed up without such efforts. Likewise the Btrfs EXT XFS file systems under Linux. So ZFS is actually supported by TSM only under Solaris.
 
I had a look into the source. Multiple subnet descriptors seem indeed currently not possible for zfs exports. The exports file syntax is created in /usr/src/cddl/compat/opensolaris/misc/fsshare.c (Rel.12) and interpreted in /usr/src/usr.sbin/mountd/mountd.c. Both are easily changeable, so if I had the issue, I would change the code. A feature request would also be appropriate. But for an evaluation task it is just a problem.
wilschie said:
Yes. IBM renamed TSM to Spectrum Protect in 2015, I think.
Click to expand...
I got tired of these renames long before that. Now I will only track them if somebody wants me to support the product. ;)

wilschie said:
ZFS on Linux is not supported by TSM. The datasets are backed up but the backup server tags the ZFS datasets / file systems as Filespace Type UNKNOWN. This may be a cosmetic issue, but it does not inspire confidence :)
Click to expand...
No it doesn't. And sadly we can't evaluate the source here to improve confidence.

wilschie said:
Furthermore, under Linux, each dataset and sub dataset to be backed up must be listed in a configuration file. This is hardly feasible with hundreds of users and fluctuation among employees and colleagues. ZFS under Solaris is backed up without such efforts. Likewise the Btrfs EXT XFS file systems under Linux. So ZFS is actually supported by TSM only under Solaris.
Click to expand...
I see. Sounds quite bad.
 
You must log in or register to reply here.
Back
Top