zirias@
Developer
The following is a mostly automated markdown-to-bbcode conversion of my new document also available here rendered directly to HTML: https://sekrit.de/webdocs/freebsd/nfs-jail-kerberos-samba.html
--------
You will find a few docs on using kerberized NFS with FreeBSD, and I recommend you read at least the KerberizedNFS FreeBSD wiki page for basic information. This document will focus on the issues specific to using a samba domain controller for kerberos authentication.
A configuration will be shown how to setup the NFS server in a jail (possibly together with a samba instance serving the same shares via SMB for Windows clients, but configuring this samba instance is out of scope).
0. Prerequisites
It is assumed you already have a samba domain controller up and running, and any machines that should access your kerberized NFS shares are joined to that domain. It is also assumed the clients are already configured to do kerberos authentication using your samba domain.
Configuration of a samba domain and FreeBSD domain members in general will be out of scope for this document.
To jail the NFS server itself, you need at least FreeBSD 13.3.
In this example, we will give the jail a ZFS dataset to manage and share. This isn't really a requirement, but you must make sure that the jail is running from its own filesystem and for NFSv4, all the shares should be available from some common mount point.
First, follow your "normal" procedure to install a new jail with at least FreeBSD 13.3. It's not described here because there are many possible ways and also different tools available to manage your jail. The important thing is: It must be a
Then, create a ZFS dataset for sharing files, e.g.
You can now create child datasets for sharing, or move existing datasets with
If you don't have a devfs ruleset to allow the ZFS device in a jail yet, create one in /etc/devfs.rules:
The name doesn't matter much, and the number should be one that's unused so far, starting with
Your interface name will most likely differ, also make sure to use the number of the devfs ruleset you just created.
This assumes you have some common settings for all jails like e.g.
Once done, issue some command like
Then also edit /etc/nsswitch.conf and /etc/pam.d/system to use account information and authentication against your domain. Again, what to put there exactly depends on your environment, e.g. whether you are using
Make sure everything works by checking some existing domain accounts with the
Finally, add these settings to your /etc/rc.conf:
2. Create an SPN for your NFS server
The NFS server will need a Service Principal Name to authenticate itself towards clients. This must be named
Log in to your samba domain controller and issue these commands:
If you have some expiration policies enabled, you might also need
A dedicated "service account" as shown here isn't strictly necessary, but best practice.
Then, export keys for your new SPN with
Transfer the
You can verify the contents of your keytab with
It should now contain at least one entry for your new SPN.
Set a mountpoint for your new
If you haven't done so yet, create some child dataset for sharing, e.g.
Now, configure /etc/exports, e.g. like this:
The first line will set the NFS root and only allow kerberos authenticated access. The second line will share our newly created dataset. You can also use the
Reboot your new file server jail, it should now start to serve NFS shares.
4. FreeBSD clients, mounting as a user
Any client must run
and start these services.
Accessing the NFS shares should immediately work from a domain-joined client for a user in posession of a valid kerberos TGT (ticket-granting ticket). You can test this on a member machine that has the
In case this does not work, you'll have to double-check your kerberos configuration. Start by inspecting the output of
A correctly working domain client will obtain a TGT on user login. It's crucial to get this working correctly, as the TGT will also be necessary to access the share even when you mount it system-wide.
5. FreeBSD clients, system-wide mounts
For system-wide mounts, we need a different way to obtain a kerberos TGT: Use a key from the local keytab as the "host based initiator". The first thing necessary for this is running
On a samba domain member, we will have keys for the AD machine account and for a "host" SPN available, e.g. like this on a machine named "client":
A Linux NFS client would automatically try the AD machine account (
As the
Trying
which will open your editor (from the
Just add a line here with a fully qualified
After saving this modified computer account, a system-wide mount using the
--------
You will find a few docs on using kerberized NFS with FreeBSD, and I recommend you read at least the KerberizedNFS FreeBSD wiki page for basic information. This document will focus on the issues specific to using a samba domain controller for kerberos authentication.
A configuration will be shown how to setup the NFS server in a jail (possibly together with a samba instance serving the same shares via SMB for Windows clients, but configuring this samba instance is out of scope).
0. Prerequisites
It is assumed you already have a samba domain controller up and running, and any machines that should access your kerberized NFS shares are joined to that domain. It is also assumed the clients are already configured to do kerberos authentication using your samba domain.
Configuration of a samba domain and FreeBSD domain members in general will be out of scope for this document.
To jail the NFS server itself, you need at least FreeBSD 13.3.
Warnings
- If you have your domain controller in a jail, do not use this same
jail for serving files (neither with NFS nor with samba)- If you want to share the files with SMB as well, or access them locally,
do not enable any "tuning" options likevfs.nfsd.issue_delegations=1
1. Create a new jailNote: All the following assumes your new file server will be named
files
, your local DNS domain islocal.example
and your kerberos realm is
LOCAL.EXAMPLE
, so replace these values as needed in all the following
commands and configurations.
In this example, we will give the jail a ZFS dataset to manage and share. This isn't really a requirement, but you must make sure that the jail is running from its own filesystem and for NFSv4, all the shares should be available from some common mount point.
First, follow your "normal" procedure to install a new jail with at least FreeBSD 13.3. It's not described here because there are many possible ways and also different tools available to manage your jail. The important thing is: It must be a
VNET
jail.Then, create a ZFS dataset for sharing files, e.g.
Code:
$ zfs create zroot/netshares
$ zfs set jailed=on zroot/netshares
zfs rename
.If you don't have a devfs ruleset to allow the ZFS device in a jail yet, create one in /etc/devfs.rules:
Code:
[devfsrules_jail_zfs=100]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'zfs' unhide
100
. Then it's time for the jail configuration, I'll show here an example what to add in plain /etc/jail.conf:
Code:
files {
vnet = new;
vnet.interface = epair1b;
allow.mount;
allow.mount.zfs;
allow.nfsd;
devfs_ruleset = 100;
enforce_statfs = 1;
exec.created = "zfs jail files zroot/netshares";
exec.release = "zfs unjail files zroot/netshares";
}
This assumes you have some common settings for all jails like e.g.
Code:
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
mount.fstab = "/var/jail/${name}.fstab";
host.hostname = "${name}.local.example";
allow.noset_hostname;
Now enter your new jail, install necessary packages (e.g.Tip: If you want to prioritize file serving on a somewhat busy host, you
might want to add something like
to the configuration of this new jail.Code:exec.start = "/usr/bin/nice -n -20 /bin/sh /etc/rc";
samba
) and join it to the domain. This requires some configuration first, so create /etc/krb5.conf and (when using samba/winbind) /usr/local/etc/smb4.conf in the same way you do on your other domain-joined machines. How exactly to configure it depends a lot on your environment, therefore this is out of scope for this document.Once done, issue some command like
net ads join -UAdministrator
(again assuming you use samba/winbind).Then also edit /etc/nsswitch.conf and /etc/pam.d/system to use account information and authentication against your domain. Again, what to put there exactly depends on your environment, e.g. whether you are using
winbind
or sssd
.Make sure everything works by checking some existing domain accounts with the
id
tool and trying to authenticate locally as some domain user.Finally, add these settings to your /etc/rc.conf:
Code:
zfs_enable="YES"
gssd_enable="YES"
nfs_server_enable="YES"
nfs_server_flags="-t"
nfsv4_server_enable="YES"
nfsv4_server_only="YES"
nfsuserd_enable="YES"
zfs_enable
is only required when using a jailed ZFS dataset as recommended here. nfs_server_flags
are necessary for a jailed NFS, because otherwise, the daemon tries to offer UDP, which doesn't work in a jail. You don't really need the nfsv4_server_only
setting, but as kerberos only works with v4, it's pointless for most scenarios to offer NFSv3. Leave it out if you want/need to share some read-only stuff to clients that can't do NFSv4, but in that case, you will also need rpcbind
.2. Create an SPN for your NFS server
The NFS server will need a Service Principal Name to authenticate itself towards clients. This must be named
nfs/<host>.<domain>
, so in our example nfs/files.local.example
.Log in to your samba domain controller and issue these commands:
Code:
$ samba-tool user create nfs-files --random-password
$ samba-tool spn add nfs/files.local.example nfs-files
Code:
$ samba-tool user setexpiry nfs-files --noexpiry
Then, export keys for your new SPN with
Code:
$ samba-tool domain exportkeytab nfs.keytab --principal=nfs/files.local.example
nfs.keytab
file to your new file server jail and add it to /etc/krb5.keytab there:
Code:
$ ktutil copy nfs.keytab /etc/krb5.keytab
Code:
$ ktutil list
Warning: Kerberos keys must be kept secret. Make sure to delete any
temporary keytab files once the keys were added where they are needed.
3. Configure your NFS sharesTip: If you ever mess up with your /etc/krb5.keytab on a domain
member, you can restore it by just deleting it and re-joining this machine
to the domain, which will initialize it with just the keys necessary for the
machine to work as a domain member.
Set a mountpoint for your new
zroot/netshares
jailed dataset:
Code:
$ zfs set mountpoint=/usr/local/netshares zroot/netshares
Code:
$ zfs create zroot/netshares/stuff
Code:
V4: /usr/local/netshares -sec=krb5:krb5i:krb5p
/usr/local/netshares/stuff
-sec
option on individual shares. Refer to the exports(5)
man page for more information.Reboot your new file server jail, it should now start to serve NFS shares.
4. FreeBSD clients, mounting as a user
Any client must run
gssd
and nfsuserd
as well, so add at least these to your /etc/rc.conf:
Code:
gssd_enable="YES"
nfsuserd_enable="YES"
Accessing the NFS shares should immediately work from a domain-joined client for a user in posession of a valid kerberos TGT (ticket-granting ticket). You can test this on a member machine that has the
vfs.usermount=1
sysctl set after logging in as a domain user:
Code:
$ mkdir mounttest # we need a mountpoint we own ourselves
$ mount -t nfs -o nfsv4,sec=krb5i files:/stuff mounttest
klist
. It should contain a TGT, and after successful access to NFS, also a service ticket for the NFS server, e.g. like:
Code:
Credentials cache: FILE:/tmp/krb5cc_10000
Principal: johndoe@LOCAL.EXAMPLE
Issued Expires Principal
Mar 21 08:13:55 2024 Mar 21 18:13:55 2024 krbtgt/LOCAL.EXAMPLE@LOCAL.EXAMPLE
Mar 21 08:13:58 2024 Mar 21 18:13:55 2024 nfs/files.local.example@LOCAL.EXAMPLE
5. FreeBSD clients, system-wide mounts
For system-wide mounts, we need a different way to obtain a kerberos TGT: Use a key from the local keytab as the "host based initiator". The first thing necessary for this is running
gssd
with the -h
flag to support this, so add the following to your /etc/rc.conf (and then restart gssd):
Code:
gssd_flags="-h"
Code:
$ ktutil list
FILE:/etc/krb5.keytab:
Vno Type Principal Aliases
10 aes256-cts-hmac-sha1-96 host/client.local.example@LOCAL.EXAMPLE
10 aes256-cts-hmac-sha1-96 host/CLIENT@LOCAL.EXAMPLE
10 aes128-cts-hmac-sha1-96 host/client.local.example@LOCAL.EXAMPLE
10 aes128-cts-hmac-sha1-96 host/CLIENT@LOCAL.EXAMPLE
10 arcfour-hmac-md5 host/client.local.example@LOCAL.EXAMPLE
10 arcfour-hmac-md5 host/CLIENT@LOCAL.EXAMPLE
10 aes256-cts-hmac-sha1-96 CLIENT$@LOCAL.EXAMPLE
10 aes128-cts-hmac-sha1-96 CLIENT$@LOCAL.EXAMPLE
10 arcfour-hmac-md5 CLIENT$@LOCAL.EXAMPLE
CLIENT$
) which always works fine. Unfortunately, FreeBSD's mount_nfs
insists on using an SPN for the purpose. There's the gssname
mount option which automatically gets /<host>.<domain>
appended, so setting it to host
here will result in host/client.local.example
.As the
host
SPN identifies the machine, it makes sense to use it, so we'll add something like this to /etc/fstab:
Code:
files:/stuff /mnt/stuff nfs nfsv4,sec=krb5i,gssname=host,late 0 0
mount /mnt/stuff
now will give you a "permission denied" error, although it should work. The reason, as far as I can tell, is that in a samba domain, an account isn't found by a service principal name attached to it, only by its sAMAccountName
or userPrincipalName
properties.What we can do about it is to edit the machine account for every client on the domain controller. These machine accounts by default don't have aWarning: I consider the following workaround for this problem a hack.
There must be a different way to solve it (and I read about a way
configuring kerberos "libdefaults" differently, but this seems to only work
with MIT krb5). If you know a way that works reliably with a FreeBSD client
using heimdal kerberos from base, please let me know, so I can update this
document.
userPrincipalName
, so we can add one. So do the following on your domain controller for the client named client
:
Code:
$ samba-tool computer edit client
EDITOR
environment variable) with all the properties of the computer account, looking something like this:
Code:
dn: CN=CLIENT,CN=Computers,DC=local,DC=example
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
objectClass: computer
cn: CLIENT
[...]
servicePrincipalName: HOST/CLIENT
servicePrincipalName: HOST/client.local.example
[...]
distinguishedName: CN=CLIENT,CN=Computers,DC=local,DC=example
userPrincipalName
"duplicating" the host
SPN (but including the kerberos realm) like this:
Code:
userPrincipalName: host/client.local.example@LOCAL.EXAMPLE
host
SPN will work on this machine.