ZFS zroot : UNAVAIL insufficient replicas...

I would like to see every file that are stored on the disk attached to that pool : da7

Code: 
# zpool status -v
  pool: nomadbsd-zroot
state: ONLINE
config:

NAME            STATE     READ WRITE CKSUM
nomadbsd-zroot  ONLINE       0     0     0
da7s2         ONLINE       0     0     0

errors: No known data errors
 
as you can see,a lot of files are missing :

Code: 
/mnt/nomadbsd-zroot/home = 47 MB
/mnt/nomadbsd-zroot/nomadbsd-zroot = 0
/mnt/nomadbsd-zroot/usr/ports = 431.7 MB
/mnt/nomadbsd-zroot/var/audit = 0
/mnt/nomadbsd-zroot/var/crash = 5 byte
/mnt/nomadbsd-zroot/var/mail = 0
 
The latest version of nomadbsd is based on freebsd 13.1 and I'm sure that it has more files than those showed on the folders mounted with the zpool import command that I've issued before.
 
Are any of those filesystems actually mounted? What does zfs mount and mount tell you?
 
Code: 
# zfs mount

nomadbsd-zroot/usr/src          /mnt/nomadbsd-zroot/usr/src
nomadbsd-zroot/home             /mnt/nomadbsd-zroot/home
nomadbsd-zroot/var/mail         /mnt/nomadbsd-zroot/var/mail
nomadbsd-zroot/var/crash        /mnt/nomadbsd-zroot/var/crash
nomadbsd-zroot/usr/obj          /mnt/nomadbsd-zroot/usr/obj
nomadbsd-zroot/var/audit        /mnt/nomadbsd-zroot/var/audit
nomadbsd-zroot                  /mnt/nomadbsd-zroot/nomadbsd-zroot
nomadbsd-zroot/usr/ports        /mnt/nomadbsd-zroot/usr/ports
nomadbsd-zroot/usr/ports/distfiles  /mnt/nomadbsd-zroot/usr/ports/distfiles
nomadbsd-zroot/usr/ports/packages  /mnt/nomadbsd-zroot/usr/ports/packages

# mount

nomadbsd-zroot/usr/src on /mnt/nomadbsd-zroot/usr/src (zfs, local, noatime, noexec, nosuid, nfsv4acls)
nomadbsd-zroot/home on /mnt/nomadbsd-zroot/home (zfs, local, noatime, nfsv4acls)
nomadbsd-zroot/var/mail on /mnt/nomadbsd-zroot/var/mail (zfs, local, noexec, nosuid, nfsv4acls)
nomadbsd-zroot/var/crash on /mnt/nomadbsd-zroot/var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
nomadbsd-zroot/usr/obj on /mnt/nomadbsd-zroot/usr/obj (zfs, local, noatime, nfsv4acls)
nomadbsd-zroot/var/audit on /mnt/nomadbsd-zroot/var/audit (zfs, local, noatime, noexec, nosuid, nfsv4acls)
nomadbsd-zroot on /mnt/nomadbsd-zroot/nomadbsd-zroot (zfs, local, noatime, nfsv4acls)
nomadbsd-zroot/usr/ports on /mnt/nomadbsd-zroot/usr/ports (zfs, local, noatime, nfsv4acls)
nomadbsd-zroot/usr/ports/distfiles on /mnt/nomadbsd-zroot/usr/ports/distfiles (zfs, local, noatime, noexec, nosuid, nfsv4acls)
nomadbsd-zroot/usr/ports/packages on /mnt/nomadbsd-zroot/usr/ports/packages (zfs, local, noatime, noexec, nosuid, nfsv4acls)
 
It worked. So,the working commands are :

Code: 
# zpool import -f -R /mnt/nomadbsd-zroot nomadbsd-zroot
# zfs mount nomadbsd-zroot/ROOT/default

I still don't know when another folder called nomadbsd-root is produced inside the path /mnt/nomadbsd-root. Is it created when I issue this command : zpool import -f -R /mnt/nomadbsd-zroot nomadbsd-zroot ?
 
ziomario said:
I still don't know when another folder called "nomadbsd-root" is produced inside the path "/mnt/nomadbsd-root". Is it created when I issue this command : "zpool import -f -R /mnt/nomadbsd-zroot nomadbsd-zroot" ?
Click to expand...
Yes.

/mnt/nomadbsd-root/ is the mountpoint you have created where the imported pool "nomadbsd-zroot" with mount point "nomadbsd-zroot" is mounted on.
Code: 
# zfs list -o name,mounted,canmount,mountpoint nomadbsd-zroot
NAME            MOUNTED  CANMOUNT  MOUNTPOINT
nomadbsd-zroot  yes      on        /mnt/nomadbsd-zroot/nomadbsd-zroot

Make a test
Code: 
# zpool export nomadbsd-zroot
# zpool import -R /mnt nomadbsd-zroot
 
I've reinstalled a new version of nomadbsd,this time on the disk da1 and I have mounted the zpool with this command :

# zpool import nomadbsd_zroot nomadbsd-zroot

It created a folder on / ,called nomadbsd-zroot and inside of it I see nothing. At this point,when I give this command :

Code: 
# zfs mount nomadbsd-zroot/ROOT/default

it mounts every folder which belongs to nomadbsd on the / of my disk and the current installation of FreeBSD stops working and reboot. I've also tried this command (because I wanted to mount everything inside the folder /nomadbsd-zroot and not in /) :

Code: 
# cd /nomadbsd-zroot
# zfs mount nomadbsd-zroot/ROOT/default .

but it gives this error :

Code: 
too many arguments
usage:
        mount
        mount [-flvO] [-o opts] <-a | filesystem>

For the property list, run: zfs set|get

For the delegated permission list, run: zfs allow|unallow
zfs list seems to be good :

Code: 
# zfs list

NAME                                           USED  AVAIL     REFER  MOUNTPOINT
nomadbsd-zroot                                2.37G   772M       96K  /nomadbsd-zroot
nomadbsd-zroot/ROOT                           2.37G   772M       96K  none
nomadbsd-zroot/ROOT/default                   2.37G   772M     2.37G  /
nomadbsd-zroot/usr                             584K   772M       96K  /usr
nomadbsd-zroot/usr/obj                          96K   772M       96K  /usr/obj
nomadbsd-zroot/usr/ports                       296K   772M      104K  /usr/ports
nomadbsd-zroot/usr/ports/distfiles              96K   772M       96K  /usr/ports/distfiles
nomadbsd-zroot/usr/ports/packages               96K   772M       96K  /usr/ports/packages
nomadbsd-zroot/usr/src                          96K   772M       96K  /usr/src
nomadbsd-zroot/var                             424K   772M       96K  /var
nomadbsd-zroot/var/audit                       104K   772M      104K  /var/audit
nomadbsd-zroot/var/crash                        96K   772M       96K  /var/crash
nomadbsd-zroot/var/mail                        128K   772M      128K  /var/mail
zroot                                          405G  40.4G       96K  /zroot
zroot/ROOT                                     393G  40.4G       96K  none
zroot/ROOT/13.1-RELEASE-p2_2022-11-11_174736     8K  40.4G      321G  /
zroot/ROOT/13.1-RELEASE-p3_2022-11-17_193042     8K  40.4G      343G  /
zroot/ROOT/13.1-RELEASE_2022-09-01_041825        8K  40.4G      308G  /
zroot/ROOT/31-03-2022-a                        512K  40.4G     43.4G  /
zroot/ROOT/31-03-2022-b                        393G  40.4G      312G  /
zroot/tmp                                     14.9M  40.4G     14.9M  /tmp
zroot/usr                                     11.2G  40.4G       96K  /usr
zroot/usr/home                                 192K  40.4G      192K  /usr/home
zroot/usr/ports                               11.2G  40.4G     11.2G  /usr/ports
zroot/usr/src-                                  96K  40.4G       96K  /usr/src-
zroot/var                                     1.26G  40.4G       96K  /var
zroot/var/audit                                 96K  40.4G       96K  /var/audit
zroot/var/crash                               1.11G  40.4G     1.11G  /var/crash
zroot/var/log                                 4.07M  40.4G     4.07M  /var/log
zroot/var/mail                                 130M  40.4G      130M  /var/mail
zroot/var/tmp                                 18.1M  40.4G     18.1M  /var/tmp
zroot2                                         348G   551G      348G  /zroot2
zroot2/ROOT                                    296K   551G       96K  none
zroot2/ROOT/default                            200K   551G      200K  /
zroot2/tmp                                      96K   551G       96K  /tmp
zroot2/usr                                     392K   551G       96K  /usr
zroot2/usr/home                                 96K   551G       96K  /usr/home
zroot2/usr/ports                               104K   551G      104K  /usr/ports
zroot2/usr/src                                  96K   551G       96K  /usr/src
zroot2/var                                    5.11M   551G       96K  /var
zroot2/var/audit                                96K   551G       96K  /var/audit
zroot2/var/crash                                96K   551G       96K  /var/crash
zroot2/var/log                                4.62M   551G     4.62M  /var/log
zroot2/var/mail                                 96K   551G       96K  /var/mail
zroot2/var/tmp                                 112K   551G      112K  /var/tmp
Code: 
# zpool export nomadbsd-zroot
cannot unmount '/usr/ports/packages': pool or dataset is busy
 
Code: 
# /bin/kenv | egrep -i "mountfrom|currdev|kernel_path|kernelname"

currdev="zfs:zroot/ROOT/31-03-2022-b:"
kernel_path="/boot/kernel"
kernelname="/boot/kernel/kernel"
vfs.root.mountfrom="zfs:zroot/ROOT/31-03-2022-b"

# zpool list -v

NAME                SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
nomadbsd-zroot     3.25G  2.37G   900M        -         -    53%    72%  1.00x    ONLINE  -
  label/nomadroot  3.25G  2.37G   900M        -         -    53%  73.0%      -    ONLINE
zroot               460G   405G  54.6G        -         -    42%    88%  1.00x    ONLINE  -
  ada0p4            460G   405G  54.6G        -         -    42%  88.1%      -    ONLINE
zroot2              928G   348G   580G        -         -     4%    37%  1.00x    ONLINE  -
  da0p4             928G   348G   580G        -         -     4%  37.5%      -    ONLINE

zroot/ROOT/31-03-2022-b         /
zroot/usr/home                  /usr/home
zroot/usr/ports                 /usr/ports
zroot/usr/src-                  /usr/src-
zroot/var/audit                 /var/audit
zroot                           /zroot
zroot/var/tmp                   /var/tmp
zroot/var/crash                 /var/crash
zroot/var/mail                  /var/mail
nomadbsd-zroot/usr/ports        /usr/ports
nomadbsd-zroot/usr/obj          /usr/obj
nomadbsd-zroot                  /nomadbsd-zroot
nomadbsd-zroot/usr/src          /usr/src
zroot2/usr/ports                /usr/ports
zroot2                          /zroot2
nomadbsd-zroot/usr/ports/distfiles  /usr/ports/distfiles
zroot2/usr/home                 /usr/home
zroot2/var/crash                /var/crash
zroot2/var/mail                 /var/mail
zroot2/var/tmp                  /var/tmp
zroot2/var/log                  /var/log
zroot/var/log                   /var/log
zroot2/tmp                      /tmp
zroot2/usr/src                  /usr/src
zroot2/var/audit                /var/audit
zroot/tmp                       /tmp
nomadbsd-zroot/usr/ports/packages  /usr/ports/packages
nomadbsd-zroot/var/mail         /var/mail
nomadbsd-zroot/var/crash        /var/crash
nomadbsd-zroot/var/audit        /var/audit
 
Please,reload the page. I've rebooted and I've updated the informations. Now it says "ONLINE".
 
I see,
Code: 
zroot/usr/ports                 /usr/ports
nomadbsd-zroot/usr/ports        /usr/ports
Which one do you want in /usr/ports? There can only be one.
The same for all other conflicting directories.

You can do on dataset-level but i disadvise:
Code: 
zfs umount /usr/ports
zfs mount nomadbsd-zroot/usr/ports

Beter import using altroot zpool import.
 
Are these commands ok ?

Code: 
# zpool export nomadbsd-zroot
# zpool import -R /mnt/nomadbsd-zroot nomadbsd-zroot
# zfs mount nomadbsd-zroot/ROOT/default
 
That looks good. All files will be under "/mnt/nomadbsd-zroot"

Code: 
zfs mount | sort -k 5
Verify Mountpoint colum all are unique. [E.g. no doubles like /usr/ports & /usr/ports ]
 
Alain De Vos said:
That looks good. All files will be under "/mnt/nomadbsd-zroot"

Code: 
zfs mount | sort -k 5
Verify Mountpoint colum all are unique. [E.g. no doubles like /usr/ports & /usr/ports ]
Click to expand...

The first command is not good :

Code: 
# zpool export nomadbsd-zroot
cannot unmount '/usr/ports/packages': pool or dataset is busy

# zfs mount | sort -k 5

nomadbsd-zroot                  /nomadbsd-zroot
nomadbsd-zroot/usr/obj          /usr/obj
nomadbsd-zroot/usr/ports        /usr/ports
nomadbsd-zroot/usr/ports/distfiles  /usr/ports/distfiles
nomadbsd-zroot/usr/ports/packages  /usr/ports/packages
nomadbsd-zroot/usr/src          /usr/src
zroot                           /zroot
zroot/ROOT/31-03-2022-b         /
zroot/tmp                       /tmp
zroot/usr/home                  /usr/home
zroot/usr/ports                 /usr/ports
zroot/usr/src-                  /usr/src-
zroot/var/audit                 /var/audit
zroot/var/crash                 /var/crash
zroot/var/log                   /var/log
zroot/var/mail                  /var/mail
zroot/var/tmp                   /var/tmp
zroot2                          /zroot2
zroot2/tmp                      /tmp
zroot2/usr/home                 /usr/home
zroot2/usr/ports                /usr/ports
zroot2/var/audit                /var/audit
zroot2/var/crash                /var/crash
zroot2/var/log                  /var/log
zroot2/var/mail                 /var/mail
zroot2/var/tmp                  /var/tmp
 
1.Nice way,
Maybe some directories are mounted and accessed.

Code: 
zfs mount | grep nomadbsd-zroot

Code: 
zfs umount  "the_entries listed above"

Then zpool export

2.Not so nice way:
Code: 
zpool export -f ...
 
Alain De Vos said:
Then zpool export
Click to expand...
A zpool-export(8) will unmount everything before exporting. But you cannot unmount a directory that's in use. This includes any shells you may have 'cd'ed into that directory.

Code: 
     Before exporting the pool,	all datasets within the	pool are unmounted.  A
     pool can not be exported if it has	a shared spare that is currently being
     used.
 
Code: 
# zfs mount | grep nomadbsd-zroot

nomadbsd-zroot/usr/obj          /usr/obj
nomadbsd-zroot/usr/src          /usr/src
nomadbsd-zroot/usr/ports        /usr/ports
nomadbsd-zroot                  /nomadbsd-zroot
nomadbsd-zroot/usr/ports/distfiles  /usr/ports/distfiles
nomadbsd-zroot/usr/ports/packages  /usr/ports/packages

# zfs umount nomadbsd-zroot/usr/obj
# zfs umount nomadbsd-zroot/usr/src 

# zfs umount nomadbsd-zroot/usr/ports
cannot unmount '/usr/ports/packages': pool or dataset is busy

# zfs umount nomadbsd-zroot/usr/ports/packages
cannot unmount '/usr/ports/packages': pool or dataset is busy
 
You must log in or register to reply here.
Back
Top