FreeBSD as NFS server(large scale)

[nizar.abed]

Hello everyone,

New to FreeBSD...
I'm considering FreeBSD as a NFS server with ZFS.
While I noticed the great features of the latest FreeBSD, including ZFS latest and, I'm trying to setup NFS server for many clients, and while tuning NFS server on FreeBSD(13.1), I couldn't change NFS maxthreads more than 256.
Setting nfs_server_flags = "-u -t -n 4000" in /etc/rc.conf
then
/etc/rc.d/nfsd restart
I get
nfsd: nfsd count too high 4000; truncated to 256

Trying to adjust under /etc/sysctl.conf
vfs.nfsd.maxthreads=4000
Also doesn't work

. is there anyway to use FreeBSD as NFS server with high number of nfs threads?
. Is FreeBSD the right choice for large scale robust NFS server?

The testing I'm doing are on pretty powerful server:
sysctl hw.model hw.machine hw.ncpu:

hw.model: AMD EPYC 7402 24-Core Processor
hw.machine: amd64
hw.ncpu: 96

768GB RAM.

Thanks in advance.

 

[peter-h]

Why not try it "as is " and slowly increasing maxthreads if performance suffer ?

 

[Phishfry]

What is the NFS max thread limit

Have you tried modifying source and test?
nfsd.c contains the variable.

Look at this posts PS too:

The nfsd.c in head/current/10.0 also has a "--maxthreads" option
that I don't think is limited to MAXNFSDCNT from a glance at the code.

rmacklem@ is active in NFS and I would ask his opinion.

 

[zirias@]

Why not try it "as is " and slowly increasing maxthreads if performance suffer ?

Follow-up question: What should "large scale" mean here?

If you'd expect up to 4000 clients and conclude you'd need 4000 threads for that, it would be a flawed assumption.

Requests don't arrive from all the clients simultaneously, so a single thread can serve multiple clients. There's also a limitation how much parallelism would work fine on your actual storage devices.

 

[Phishfry]

I think it would be worth exploring pNFS(4) as well. (Parallel NFS Protocol)
It just hit in 12-RELEASE and was upgraded in 13-RELEASE.

https://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt

It does go against your 'one big NFS machine' philosophy. Distributes the load with parallel servers.

http://www.pnfs.com/

 

[Sergei_Shablovsky]

Hello everyone,

New to FreeBSD...
I'm considering FreeBSD as a NFS server with ZFS.

. is there anyway to use FreeBSD as NFS server with high number of nfs threads?
. Is FreeBSD the right choice for large scale robust NFS server?

Why not to use well-reputable FreeNAS ? Also based on FreeBSD...

The testing I'm doing are on pretty powerful server:
sysctl hw.model hw.machine hw.ncpu:

hw.model: AMD EPYC 7402 24-Core Processor
hw.machine: amd64
hw.ncpu: 96
768GB RAM.

Really powerful machine.

But for NAS You need care more on:
- best possible RAID/hub card;
- 2 PSU in your server case;
- great online interactive UPS (like Liebert or Emerson made, rack(!) version);
- a lot slots of 3,5” SATA drives;
- a lot of ECC RAM (ChipKill is also great choice);
- better to have active backplane enhancer;

 

[Phishfry]

Why not to use well-reputable FreeNAS ? Also based on FreeBSD...

If this guy is managing 4K NFS seats I doubt he needs a web interface.