zfs zroot diskspace puzzle

cbrace

Well-Known Member

Reaction score: 16
Messages: 319

Hi all,
I have a simple LAN file server running v12.2. The OS is installed on a 120GB SSD using the ZFS defaults:
Code:
$ sudo gpart show /dev/ada4
=>       40  234441568  ada4  GPT  (112G)
         40       1024     1  freebsd-boot  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  freebsd-swap  (2.0G)
    4196352  230244352     3  freebsd-zfs  (110G)
  234440704        904        - free -  (452K)
I would like to upgrade this to v12.3 shortly. But I seem to have run out of diskspace; bectl cannot create a boot image. In the info below, I see that zroot/ROOT appears to be using 96.5G, but I cannot determine exactly how this space is being used, and how to free it up. I have some earlier BE's still online, which I should probably delete, but these snapshots don't appear to be using that much space, maybe around 7G.

Any ideas for figuring out what is taking up so much space on my zroot/ROOT?
Code:
NAME                          USED  AVAIL  REFER  MOUNTPOINT
zroot                         100G  5.40G    88K  /zroot
zroot/ROOT                   96.5G  5.40G    88K  none
zroot/ROOT/12.1-RELEASE       212K  10.0G  1.63G  /
zroot/ROOT/12.1-RELEASE-p1    720K  10.0G  3.16G  /
zroot/ROOT/12.1-RELEASE-p10   864K  10.0G  3.56G  /
zroot/ROOT/12.1-RELEASE-p9    684K  10.0G  3.20G  /
zroot/ROOT/12.2-RELEASE       852K  10.0G  4.45G  /
zroot/ROOT/12.2-RELEASE-p1    624K  10.0G  4.44G  /
zroot/ROOT/12.2-RELEASE-p10  12.5G  5.40G  5.38G  /
zroot/ROOT/12.2-RELEASE-p2    980K  12.4G  4.57G  /
zroot/ROOT/12.2-RELEASE-p3    804K  12.4G  4.63G  /
zroot/ROOT/12.2-RELEASE-p4    464K  12.4G  4.71G  /
zroot/ROOT/12.2-RELEASE-p5    568K  12.4G  4.74G  /
zroot/ROOT/12.2-RELEASE-p6    760K  12.4G  4.94G  /
zroot/ROOT/12.2-RELEASE-p7    764K  12.4G  5.18G  /
zroot/tmp                    11.5M   489M  11.5M  /tmp
zroot/usr                    3.58G  5.40G    88K  /usr
zroot/usr/home               3.58G  5.40G  3.58G  /usr/home
zroot/var                    13.9M  5.40G    88K  /var
zroot/var/log                13.7M  36.3M  13.7M  /var/log
zroot/var/mail                104K   920K   104K  /var/mail

Code:
$ zfs list -o space -t all -S usedds | head
NAME                                                       AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
zroot/ROOT/12.2-RELEASE-p10                                5.40G  12.5G     7.16G   5.38G              0          0
zroot/usr/home                                             5.40G  3.58G         0   3.58G              0          0
zroot/var/log                                              36.3M  13.7M         0   13.7M              0          0
zroot/tmp                                                   489M  11.5M         0   11.5M              0          0
zroot/ROOT/12.2-RELEASE-p2                                 12.4G   980K         0    980K              0          0
zroot/ROOT/12.1-RELEASE-p10                                10.0G   864K         0    864K              0          0
zroot/ROOT/12.2-RELEASE                                    10.0G   852K         0    852K              0          0
zroot/ROOT/12.2-RELEASE-p3                                 12.4G   804K         0    804K              0          0
 

Alain De Vos

Son of Beastie

Reaction score: 999
Messages: 3,085

You could have a look:
Code:
mkdir /test
mount -t zfs zroot/ROOT /test
du -hs /test/*
 

Geezer

Aspiring Daemon

Reaction score: 618
Messages: 938

What are all these?
Code:
zroot/ROOT/12.1-RELEASE       212K  10.0G  1.63G  /
zroot/ROOT/12.1-RELEASE-p1    720K  10.0G  3.16G  /
zroot/ROOT/12.1-RELEASE-p10   864K  10.0G  3.56G  /
zroot/ROOT/12.1-RELEASE-p9    684K  10.0G  3.20G  /
zroot/ROOT/12.2-RELEASE       852K  10.0G  4.45G  /
zroot/ROOT/12.2-RELEASE-p1    624K  10.0G  4.44G  /
zroot/ROOT/12.2-RELEASE-p10  12.5G  5.40G  5.38G  /
zroot/ROOT/12.2-RELEASE-p2    980K  12.4G  4.57G  /
zroot/ROOT/12.2-RELEASE-p3    804K  12.4G  4.63G  /
zroot/ROOT/12.2-RELEASE-p4    464K  12.4G  4.71G  /
zroot/ROOT/12.2-RELEASE-p5    568K  12.4G  4.74G  /
zroot/ROOT/12.2-RELEASE-p6    760K  12.4G  4.94G  /
zroot/ROOT/12.2-RELEASE-p7    764K  12.4G  5.18G  /

All on the same mountpoint, and each using up a few G.
 

Eric A. Borisch

Aspiring Daemon

Reaction score: 389
Messages: 616

Code:
$ zfs list -o space -t all -S usedds | head
NAME                                                       AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
zroot/ROOT/12.2-RELEASE-p10                                5.40G  12.5G     7.16G   5.38G              0          0
zroot/usr/home                                             5.40G  3.58G         0   3.58G              0          0
zroot/var/log                                              36.3M  13.7M         0   13.7M              0          0
zroot/tmp                                                   489M  11.5M         0   11.5M              0          0
zroot/ROOT/12.2-RELEASE-p2                                 12.4G   980K         0    980K              0          0
zroot/ROOT/12.1-RELEASE-p10                                10.0G   864K         0    864K              0          0
zroot/ROOT/12.2-RELEASE                                    10.0G   852K         0    852K              0          0
zroot/ROOT/12.2-RELEASE-p3                                 12.4G   804K         0    804K              0          0

How about the full output from zfs list -ro space zroot/ROOT ?
 

mer

Daemon

Reaction score: 667
Messages: 1,035

I would start off with deleting all of the BEs except for the one in use and maybe the one previous.
 

grahamperrin

Beastie's Twin

Reaction score: 1,713
Messages: 5,093

… don't appear to be using that much space, maybe around 7G. …

I would start off with deleting all of the BEs except for the one in use and maybe the one previous.

It's not yet clear how much free space is expected now, or after taking action, which is partly why I'd like to see output from zpool list.
 

mer

Daemon

Reaction score: 667
Messages: 1,035

Regardless of how it looks, removing boot environments will clean things up and make it easier to figure out where the space is actually being used.
Why do I say this? Boot Environments are clones (writable snapshots basically) and the space allocation gets a little weird to understand because of the COW behavior.

I'm not arguing that full output is not needed to help the OP, obviously we need to see it all. A listing of snapshots from the top level would also be useful because perhaps the OP has some snapshots he's forgotten about somewhere.
 
Top