Private FreeBSD server with vms, jails, AD DC and network shares for different client OS

zirias@

Developer
I'm running into a lot of dead ends here and looking for some tips what else to try, so I'll first describe my scenario:

For my house, I built a single server (as this is a private home, it's important that it's a single real machine with acceptable power consumption). The server should provide (at least) storage, authentication and decent security to several client machines. The current setup looks like this:
  • Host system: FreeBSD 11.1 built without kerberos and a few other things and a kernel with VIMAGE enabled. This system manages storage (ZFS pool) and some jails and bhyve vms that get network connectivity using bridges (with tap(4) members for vms and epair(4) members for jails). On the bridge meant for the management network, the host itself has IPv4 configured.
  • A bhyve virtual machine also running (a minimal) FreeBSD 11.1 that gets the physical NICs by PCI passthru and is plugged in all the bridges, as the central router and firewall. It connects to my switch through lagg(4) with several VLAN devices on top.
  • A jail for building packages with poudriere
  • A jail running nginx as the internal webserver, currently only serving the package repository
  • A jail running Samba 4.8 as AD DC
  • A jail also running Samba 4.8 as domain member, serving user / home directories, windows profiles and a common shared directory
There's a bit more already, but that's probably not important here. This setup already works fine for Windows 10 Clients. I also got it to work with a Linux client, using Linux' CIFS VFS for mounting home directories with pam_mount, after installling Samba on the Linux machine and joining it to the domain. Now I installed FreeBSD 11.1 on my Desktop machine as well and that's where the problems begin :( Starting point here is an installation with winbind working for NSS and PAM (Samba 4.8 is installed and joined to the domain)

I didn't use NFS so far because it won't work in a jail -- it would need to run on the host system, and I didn't plan to allow network connections from my internal network to it. So I tried anything to get my home directories with mount_smbfs(8) with no luck at all. It's always giving me authentication errors. Does it even work in an active directory scenario?

I also tried running net/unfs3 in a jail. This seemed to work, but exposes strange behavior: Trying to access the files as a non-root user always gives "permission denied", although the IDs are correct.

Now I'm thinking about enabling the NFS server on the host system, and have the router/firewall VM forward all NFS requests -- could this actually work? Does anybody have a better idea?
 
Now I'm thinking about enabling the NFS server on the host system, and have the router/firewall VM forward all NFS requests -- could this actually work?
Just in case anyone's interested, this does work.

I added the following on my host system to /etc/rc.conf:
Code:
rpcbind_enable="YES"
nfs_server_enable="YES"
mountd_flags="-p33333"
The mountd flag makes sure a fixed port is used. Then, on the firewall vm, I added this to /etc/pf.conf:
Code:
int_if = "bridge1"

localnet = $int_if:network

nfsvirt = "192.168.42.103"
nfsreal = "192.168.17.101"
nfsports = "{111,2049,33333}"

rdr on $int_if inet proto tcp from $localnet \
    to $nfsvirt port $nfsports -> $nfsreal
rdr on $int_if inet proto udp from $localnet \
    to $nfsvirt port $nfsports -> $nfsreal
192.168.42.103 is the IP address of my file server jail, 192.168.17.101 the IP address of the host system on the management network. This redirect allows the client to mount "from the file server" although the nfs service is running directly on the host. Doesn't look very elegant, but at least it works :)
 
You should be able to mount an nfs share in a secure fashion within a jail, by first mounting the nfs share to the host system and than expose it using the jails /etc/fstab.jail_name file just like you would mount any other host folder within a jail.

add this to /etc/fstab.jail_name to mount the nfs share after you have mounted the share to the host

Code:
/hosts/nfs/folder /usr/jails/fstab.jail_name/some/folder/in.jail nullfs rw 1 1
 
Back
Top