Solved How to extend zfs storage filesystem...

ziomario · Dec 7, 2022

Hello.

I'm trying to learn how to extend the zfs storage filesystem of nomadBSD that I have installed on the disk da1. It's size is 298 GB ; the filesystem size of nomadBSD is 3.3G but I have 295 Gb left on the disk :

Code:

# gpart show

=>       63  625142385  da1  MBR  (298G)
         63       1985       - free -  (993K)
       2048      81920    1  efi  (40M)
      83968    7022592    2  freebsd  [active]  (3.3G)
    7106560  618035888       - free -  (295G)

According with this site,where the OP asked the same :

Extend zfs storage filesystem

Hello, Need to ask the question regarding extending the zfs storage file system. currently after using the command, df -kh u01-data-pool/data 600G 552 48G 93% /data /data are only 48 gb remaining and | The UNIX and Linux Forums

www.unix.com

this is how is my situation :

Code:

# zfs list

NAME                                           USED  AVAIL     REFER  MOUNTPOINT
nomadbsd-zroot                                2.37G   772M       96K  /mnt/nomadbsd-zroot/nomadbsd-z
root
nomadbsd-zroot/ROOT                           2.37G   772M       96K  none
nomadbsd-zroot/ROOT/default                   2.37G   772M     2.37G  /mnt/nomadbsd-zroot
nomadbsd-zroot/usr                             584K   772M       96K  /mnt/nomadbsd-zroot/usr
nomadbsd-zroot/usr/obj                          96K   772M       96K  /mnt/nomadbsd-zroot/usr/obj
nomadbsd-zroot/usr/ports                       296K   772M      104K  /mnt/nomadbsd-zroot/usr/ports
nomadbsd-zroot/usr/ports/distfiles              96K   772M       96K  /mnt/nomadbsd-zroot/usr/ports/
distfiles
nomadbsd-zroot/usr/ports/packages               96K   772M       96K  /mnt/nomadbsd-zroot/usr/ports/
packages
nomadbsd-zroot/usr/src                          96K   772M       96K  /mnt/nomadbsd-zroot/usr/src
nomadbsd-zroot/var                             424K   772M       96K  /mnt/nomadbsd-zroot/var
nomadbsd-zroot/var/audit                       104K   772M      104K  /mnt/nomadbsd-zroot/var/audit
nomadbsd-zroot/var/crash                        96K   772M       96K  /mnt/nomadbsd-zroot/var/crash
nomadbsd-zroot/var/mail                        128K   772M      128K  /mnt/nomadbsd-zroot/var/mail
zroot                                          405G  40.3G       96K  /zroot
zroot/ROOT                                     393G  40.3G       96K  none
zroot/ROOT/13.1-RELEASE-p2_2022-11-11_174736     8K  40.3G      321G  /
zroot/ROOT/13.1-RELEASE-p3_2022-11-17_193042     8K  40.3G      343G  /
zroot/ROOT/13.1-RELEASE_2022-09-01_041825        8K  40.3G      308G  /
zroot/ROOT/31-03-2022-a                        512K  40.3G     43.4G  /
zroot/ROOT/31-03-2022-b                        393G  40.3G      312G  /
zroot/tmp                                     14.9M  40.3G     14.9M  /tmp
zroot/usr                                     11.2G  40.3G       96K  /usr
zroot/usr/home                                 192K  40.3G      192K  /usr/home
zroot/usr/ports                               11.2G  40.3G     11.2G  /usr/ports
zroot/usr/src-                                  96K  40.3G       96K  /usr/src-
zroot/var                                     1.26G  40.3G       96K  /var
zroot/var/audit                                 96K  40.3G       96K  /var/audit
zroot/var/crash                               1.11G  40.3G     1.11G  /var/crash
zroot/var/log                                 3.89M  40.3G     3.89M  /var/log
zroot/var/mail                                 130M  40.3G      130M  /var/mail
zroot/var/tmp                                 18.1M  40.3G     18.1M  /var/tmp
zroot2                                         348G   551G      348G  /zroot2
zroot2/ROOT                                    296K   551G       96K  none
zroot2/ROOT/default                            200K   551G      200K  /
zroot2/tmp                                      96K   551G       96K  /tmp
zroot2/usr                                     392K   551G       96K  /usr
zroot2/usr/home                                 96K   551G       96K  /usr/home
zroot2/usr/ports                               104K   551G      104K  /usr/ports
zroot2/usr/src                                  96K   551G       96K  /usr/src
zroot2/var                                    5.11M   551G       96K  /var
zroot2/var/audit                                96K   551G       96K  /var/audit
zroot2/var/crash                                96K   551G       96K  /var/crash
zroot2/var/log                                4.62M   551G     4.62M  /var/log
zroot2/var/mail                                 96K   551G       96K  /var/mail
zroot2/var/tmp                                 112K   551G      112K  /var/tmp

# zfs list -t all -o space -r nomadbsd-zroot

NAME                                AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
nomadbsd-zroot                       772M  2.37G        0B     96K             0B      2.37G
nomadbsd-zroot/ROOT                  772M  2.37G        0B     96K             0B      2.37G
nomadbsd-zroot/ROOT/default          772M  2.37G        0B   2.37G             0B         0B
nomadbsd-zroot/usr                   772M   584K        0B     96K             0B       488K
nomadbsd-zroot/usr/obj               772M    96K        0B     96K             0B         0B
nomadbsd-zroot/usr/ports             772M   296K        0B    104K             0B       192K
nomadbsd-zroot/usr/ports/distfiles   772M    96K        0B     96K             0B         0B
nomadbsd-zroot/usr/ports/packages    772M    96K        0B     96K             0B         0B
nomadbsd-zroot/usr/src               772M    96K        0B     96K             0B         0B
nomadbsd-zroot/var                   772M   424K        0B     96K             0B       328K
nomadbsd-zroot/var/audit             772M   104K        0B    104K             0B         0B
nomadbsd-zroot/var/crash             772M    96K        0B     96K             0B         0B
nomadbsd-zroot/var/mail              772M   128K        0B    128K             0B         0B

# zpool list nomadbsd-zroot

NAME             SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
nomadbsd-zroot  3.25G  2.37G   900M        -         -    53%    72%  1.00x    ONLINE  /mnt/nomadbsd
-zroot

# zfs get refquota,quota,refreservation,reservation nomadbsd-zroot

NAME            PROPERTY        VALUE      SOURCE
nomadbsd-zroot  refquota        none       default
nomadbsd-zroot  quota           none       default
nomadbsd-zroot  refreservation  none       default
nomadbsd-zroot  reservation     none       default

According with the post,the command to issue could be :

Code:

# zfs set quota=290g nomadbsd-zroot

but I'm not sure,because the file system nomadbsd-zroot has a reservation property set to none,while on the example explained in the website it is set to 200G. So,I presume that I can't do it so easily. I would like to gather more informations before issuing random commands. Anyway I tried and it seems that a part of the problem has been solved ?

Code:

# zfs set quota=290g nomadbsd-zroot
# zfs get refquota,quota,refreservation,reservation nomadbsd-zroot

NAME            PROPERTY        VALUE      SOURCE
nomadbsd-zroot  refquota        none       default
nomadbsd-zroot  quota           290G       local
nomadbsd-zroot  refreservation  none       default
nomadbsd-zroot  reservation     none       default

what to do now ???

Alain De Vos · Dec 7, 2022

zpool-add.8 — OpenZFS documentation

openzfs.github.io

ziomario · Dec 7, 2022

I need a more detailed help. There are so many variables that aren't taken into account on the man pages and I can't exclude more of them because my low level of theory and experience. Helping like you did is totally not useful for a newbie that are trying now to learn for the first time how to implement an advanced task like that.

Alain De Vos · Dec 7, 2022

Trying to understand man pages is a good exercise.
Let's say you want to extend zpool nomadx with 10G of space.
You create sparse file

Code:

truncate -s 10G /root/extend_nomadx

You add it to zpool nomadx

Code:

zpool add nomadx /root/extend_nomadx

Now when you do

Code:

zpool list -v

zpool nomadx will be 10G larger in total. Data will be striped.

ziomario · Dec 7, 2022

I don't agree. When a manual is too technical,you don't know how to start. It seems that you think that the understanding depends about the will. That's a false beliefe. Or it is very partial. Anwyay I stop here. Because talking about psychology (which is my real competence area) here does not match. So,If I have understood correctly,the correct commands to give are the following :

Code:

# truncate -s 280G /root/extend_nomadbsd-zrootx
# zpool add nomadbsd-zroot /root/extend_nomadbsd-zrootx

Anyway,I'm not sure that it does what I want.

Alain De Vos · Dec 7, 2022

I've been a teacher. So at first things are always hard. There is a mountain you must climb. And we all try at are own pace.
The commands seem correct.
Please note after adding you can't remove anymore. Data is striped. Try first with a small file. Otherwise you lose disk space.
The only thing is a backup and make a zpool from scratch with the good size & copy-over.

ziomario · Dec 7, 2022

If you have been a teacher you know for sure that you can talk like that to one of your student but not to a farmer or to a nurser and so on ? When the competence areas are so different,you can't ask to someone to understand something that requires a lot of preliminary informations. If it is not like that,what's the sense to have teachers in the world ?

Alain De Vos · Dec 7, 2022

I was a teacher with patience. As a teacher you must always have patience. At the same time ask have you tried to climb this mountain. Offcourse without knocking with a hammer.

ziomario · Dec 7, 2022

Teachers have the task of chewing knowledge to make it more usable for their students. They don't give them a manual full of technicalities and tell them to figure out what to do.

Alain De Vos · Dec 7, 2022

Trust me i am not capable of adding C-code to the freebsd-kernel. And i probably never won't be able. So i try to climb smaller mountains.
We don't live in a perfect world. As human we are born in-perfect. But that makes us also human.

ziomario · Dec 7, 2022

Code:

# truncate -s 280G /zroot2/extend_nomadbsd-zrootx
# zpool add nomadbsd-zroot /zroot2/extend_nomadbsd-zrootx

invalid vdev specification
use '-f' to override the following errors:
mismatched replication level: pool uses disk and new vdev is file

so,maybe like this ?

Code:

zpool -f add nomadbsd-zroot /zroot2/extend_nomadbsd-zrootx

Alain De Vos · Dec 7, 2022

To be honest i don't know.
You could try first

Code:

zpool labelclear -f /zroot2/extend_nomadbsd-zrootx
zpool add nomadbsd-zroot /zroot2/extend_nomadbsd-zrootx

Otherwise

Code:

zpool add -f nomadbsd-zroot /zroot2/extend_nomadbsd-zrootx

Alain De Vos · Dec 7, 2022

I just wrote a small script you might like it,

Code:

#!/usr/local/bin/zsh -x
zpool export myzpool
rm /root/myzpool
truncate -s 10G /root/myzpool
zpool create -m /mnt/myzpool -f myzpool /root/myzpool
zpool export myzpool
zpool import -d /root/myzpool myzpool
touch /mnt/myzpool/afile
zfs snapshot myzpool@now
zfs send -R myzpool@now > /root/myzpool_snapshot
zfs umount myzpool
zpool export myzpool
zpool labelclear -f /root/myzpool
zpool labelclear -f /root/myzpool_extend
rm /root/myzpool
rm /root/myzpool_extend
truncate -s 10G /root/myzpool
truncate -s 10G /root/myzpool_extend
zpool create -m /mnt/myzpool -f myzpool /root/myzpool
zpool add myzpool /root/myzpool_extend
zfs receive -F -d myzpool < /root/myzpool_snapshot
du -h myzpool
du -h myzpool_exend

ziomario · Dec 7, 2022

For sure I need to execute the commands one by one and not inside a script

. Anyway I've adapted them to my scenario. Not sure if I have "rephrased" every command correctly.

Code:

zpool export nomadbsd-zroot
rm /mnt/nomadbsd-zroot
truncate -s 280G /zroot2/nomadbsd-zroot
zpool create -m /mnt/nomadbsd-zroot -f nomadbsd-zroot /zroot2/nomadbsd-zroot
zpool export nomadbsd-zroot
zpool import -d /zroot2/nomadbsd-zroot nomadbsd-zroot
touch /mnt/nomadbsd-zroot/afile
zfs snapshot nomadbsd-zroot@now
zfs send -R nomadbsd-zroot@now > /zroot2/nomadbsd-zroot_snapshot
zfs umount nomadbsd-zroot
zpool export nomadbsd-zroot
zpool labelclear -f /zroot2/nomadbsd-zroot
zpool labelclear -f /zroot2/nomadbsd-zroot_extend
rm /zroot2/nomadbsd-zroot
rm /zroot2/nomadbsd-zroot_extend
truncate -s 280G /zroot2/nomadbsd-zroot
truncate -s 280G /zroot2/nomadbsd-zroot_extend
zpool create -m /mnt/nomadbsd-zroot -f nomadbsd-zroot /zroot2/nomadbsd-zroot
zpool add nomadbsd-zroot /zroot2/nomadbsd-zroot_extend
zfs receive -F -d nomadbsd-zroot < /zroot2/nomadbsd-zroot_snapshot
du -h nomadbsd-zroot
du -h nomadbsd-zroot_exend

Alain De Vos · Dec 7, 2022

More related to nomadbsd i found an interesting link,

https://raw.githubusercontent.com/nomadbsd/NomadBSD/master/pkg.list

freebuser · Dec 7, 2022

Haven't read the whole issue, but if you have to use -f always double, triple,quadruple check if everything is correct before issuing the command.

There is almost always something not right if there is an error output.

tyson · Dec 7, 2022

ziomario said:

For sure I need to execute the commands one by one and not inside a script

. Anyway I've adapted them to my scenario. Not sure if I have "rephrased" every command correctly.

Code:

zpool export nomadbsd-zroot
rm /mnt/nomadbsd-zroot
truncate -s 280G /zroot2/nomadbsd-zroot
zpool create -m /mnt/nomadbsd-zroot -f nomadbsd-zroot /zroot2/nomadbsd-zroot
zpool export nomadbsd-zroot
zpool import -d /zroot2/nomadbsd-zroot nomadbsd-zroot
touch /mnt/nomadbsd-zroot/afile
zfs snapshot nomadbsd-zroot@now
zfs send -R nomadbsd-zroot@now > /zroot2/nomadbsd-zroot_snapshot
zfs umount nomadbsd-zroot
zpool export nomadbsd-zroot
zpool labelclear -f /zroot2/nomadbsd-zroot
zpool labelclear -f /zroot2/nomadbsd-zroot_extend
rm /zroot2/nomadbsd-zroot
rm /zroot2/nomadbsd-zroot_extend
truncate -s 280G /zroot2/nomadbsd-zroot
truncate -s 280G /zroot2/nomadbsd-zroot_extend
zpool create -m /mnt/nomadbsd-zroot -f nomadbsd-zroot /zroot2/nomadbsd-zroot
zpool add nomadbsd-zroot /zroot2/nomadbsd-zroot_extend
zfs receive -F -d nomadbsd-zroot < /zroot2/nomadbsd-zroot_snapshot
du -h nomadbsd-zroot
du -h nomadbsd-zroot_exend

Am I too stupid or is it ZFS pool backed by files ? Wonder why even ppl would try to do it this way.

Ps. I didn’t read whole thread because I’m lazy.

Alain De Vos · Dec 7, 2022

Feel free,

ZFS - zpool on a file

You can create a zpool on a file. But why want you do so. I.e. what are the good use cases ?

forums.freebsd.org

ziomario · Dec 7, 2022

Alain De Vos said:
More related to nomadbsd i found an interesting link,

https://raw.githubusercontent.com/nomadbsd/NomadBSD/master/pkg.list

why do you find that link interesting ?

ziomario · Dec 7, 2022

It seems to me that the method suggested is not the most used and accepted. Anyway thanks for your help. I wait that someone else suggests something different.

Alain De Vos · Dec 7, 2022

ziomario said:
why do you find that link interesting ?

Nomadisize your freebsd install. Take the best of both worlds.

ziomario · Dec 7, 2022

The idea is nice,but it conflicts with the project that I'm trying to accomplish at this time.

ziomario · Dec 7, 2022

It seems that it worked...

Code:

# zpool add -f nomadbsd-zroot /zroot2/extend_nomadbsd-zrootx

# zfs list


NAME                                           USED  AVAIL     REFER  MOUNTPOINT

nomadbsd-zroot                                2.37G   270G       96K  /mnt/nomadbsd-zroot/nomadbsd-zroot
nomadbsd-zroot/ROOT                           2.37G   270G       96K  none
nomadbsd-zroot/ROOT/default                   2.37G   270G     2.37G  /mnt/nomadbsd-zroot
nomadbsd-zroot/usr                             584K   270G       96K  /mnt/nomadbsd-zroot/usr
nomadbsd-zroot/usr/obj                          96K   270G       96K  /mnt/nomadbsd-zroot/usr/obj
nomadbsd-zroot/usr/ports                       296K   270G      104K  /mnt/nomadbsd-zroot/usr/ports
nomadbsd-zroot/usr/ports/distfiles              96K   270G       96K  /mnt/nomadbsd-zroot/usr/ports/distfiles
nomadbsd-zroot/usr/ports/packages               96K   270G       96K  /mnt/nomadbsd-zroot/usr/ports/packages
nomadbsd-zroot/usr/src                          96K   270G       96K  /mnt/nomadbsd-zroot/usr/src
nomadbsd-zroot/var                             424K   270G       96K  /mnt/nomadbsd-zroot/var
nomadbsd-zroot/var/audit                       104K   270G      104K  /mnt/nomadbsd-zroot/var/audit
nomadbsd-zroot/var/crash                        96K   270G       96K  /mnt/nomadbsd-zroot/var/crash
nomadbsd-zroot/var/mail                        128K   270G      128K  /mnt/nomadbsd-zroot/var/mail

ziomario · Dec 8, 2022

You gave me a great help,teacher. Very thanks.