jails NFS sever, cannot export any kind of mounted filesystem, nor NullFS nor ZFS either guest mounted or host mounted

F

Ferocious1881

Replication steps:
1. Get an fresh FreeBSD jail instalation with bsdinstall jail at /jails/nfs.
2. Ran jail -crm -f jail.conf, where the file content is
Code: 
nfs {
exec.clean;
#exec.start  = "zfs mount zroot/jroh/nfs-jailed";
exec.start += "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown jail";
exec.consolelog = "/var/log/jail_console_${name}.log";

allow.raw_sockets;
allow.nfsd;
allow.mount;
allow.mount.devfs;
allow.mount.zfs;
mount.devfs;
enforce_statfs = 1;
devfs_ruleset = 5;

path = "/jails/${name}";
host.hostname = "${name}";

$id = "6";
$ip = "192.168.0.${id}/24";
$gateway = "192.168.0.1";
$bridge = "bridge0";
$epair = "epair${id}";

vnet;
vnet.interface = "${epair}b";

exec.created  += "zfs jail nfs zroot/jroh/nfs-jailed";
exec.prestart  = "/sbin/ifconfig ${epair} create up";
exec.prestart += "/sbin/ifconfig ${epair}a up descr jail:${name}";
exec.prestart += "/sbin/ifconfig ${bridge} addm ${epair}a up";
exec.start    += "/sbin/ifconfig ${epair}b ${ip} up";
exec.start    += "/sbin/route add default ${gateway}";
exec.poststop = "/sbin/ifconfig ${bridge} deletem ${epair}a";
exec.poststop += "/sbin/ifconfig ${epair}a destroy";
#exec.release  += "zfs unjail nfs zroot/codata/nfs";
}
4. Set the /etc/rc.conf of the jail to
Code: 
moused_nondefault_enable="NO"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
# dumpdev="NO"

nfs_server_enable="YES"
nfsv4_server_enable="YES"
nfsuserd_enable="YES"
nfs_server_flags="-t -n 4"
rpcbind_enable="YES"
mountd_flags="-r"
mountd_enable="YES"

rpc_lockd_enable="YES"
5. Run
Code: 
cat | jexec nfs tee /etc/exports <<EOF
V4: /jroh/nfs-jailed/
/jroh/nfs-jailed/probe1 -mapall=root -sec=krb5p
# Some comment
/jroh/nfs-jailed/probe2 -mapall=root 192.168.0.201 # IP of the linux machine
/jroh/nfs-jailed/probe2 -mapall=root 192.168.0.226 # IP of the bridge
/jroh/nfs-jailed/probe3 10.10.10.10
EOF
6. run
Code: 
jexec nfs service rpcbind enable
jexec nfs service nfsd enable
jexec nfs service mountd enable
jexec nfs service rpcbind status || jexec nfs service rpcbind start
jexec nfs service nfsd status || jexec nfs service nfsd start
jexec nfs service mountd status || jexec nfs service mountd start
7. run jexec nfs showmount -e and get
Code: 
Exports list on localhost:
/jroh/nfs-jailed/probe3            10.10.10.10
/jroh/nfs-jailed/probe2            192.168.0.201 192.168.0.226
/jroh/nfs-jailed/probe1            Everyone
8. Try to mount your NFS share with mount -t nfs nfs.my.domain:/jroh/nfs-jailed/probe2 /mnt and get
Code: 
mount_nfs: nmount: /mnt: Permission denied
9. Go tho the Linux machine and try the same mount and get
Code: 
mount.nfs: mounting nfs.my.domain:/jroh/nfs-jailed/probe2 failed, reason given by server: No such file or directory
[/cmd]
 
Last edited:
I have found in the logs the following error, which correlates with attempting to mount the nfs share in the a FreeBSD client:
Code: 
# /var/log/messages
Oct  3 02:07:32 my-physical-box kernel: newnfs: server 'nfs.my.domain' error: fileid changed. fsid 0:0: expected fileid 0x2, got 0x22. (BROKEN NFS SERVER OR MIDDLEWARE)
 
Tried with a \etc\exports:
Code: 
V4: /jroh/nfs-jailed/
/jroh/nfs-jailed/probe1 -mapall=root -sec=krb5p
/jroh/nfs-jailed/probe2 -mapall=root 192.168.0.201
/jroh/nfs-jailed/probe2 -mapall=root 192.168.0.226
/jroh/nfs-jailed/probe3 10.10.10.10
/ 192.168.0.226 192.168.0.201
and tried to mount it with mount -t nfs nfs.my.domain:/ /mnt in both my FreeBSD host device, with the output:
Code: 
mount_nfs: nmount: /mnt: Permission denied

Nobody, have any idea of what is happening?
 
Have recreated the jail in a new filesystem, because
Ferocious1881 said:
Have the problem again:
I confirmed to not have mount with mount | grep /jails/nfs, then make an index find /jails/nfs > index and check for open files cat index | xargs -L1 fstat | uniq
And got the following report
Code: 
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W NAME
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): Device busy
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: kinfo_getfile(): No such process
fstat: /jails/nfs/var/db/etcupdate/current/usr/share/nls/en_US.US_ASCII: No such file or directory
fstat: /jails/nfs/var/db/etcupdate/current/usr/share/nls/POSIX: No such file or directory
fstat: /jails/nfs/var/db/etcupdate/current/etc/os-release: No such file or directory
fstat: /jails/nfs/var/db/etcupdate/current/etc/rmt: No such file or directory
fstat: /jails/nfs/var/db/etcupdate/current/etc/unbound: No such file or directory
fstat: /jails/nfs/sys: No such file or directory
Click to expand...
and disabling ZFS, and there is not problem, now have to reenable ZFS. Renabling ZFS does cause problems.

For what I see the problem only appears when I mount a jailed zfs from inside of the Jail. Does not matter what.

Nor makes exports if I do mount nullfs the filesystems from another zfs mount I don't export the system. I have to say for sue that I may do a problem report. Regardless if is in a fstab
 
You must log in or register to reply here.
Back
Top