Mount zroot/usr/home on another disc.

[judd]

Good afternoon.

I want to mount zroot/usr/home to the second 500 GB HHD disk of the same laptop where FBSD 12.1 is installed on a 30 GB SSD:

# lsblk
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:97  466G GPT                               - -
  <FREE>         -:-   1.0M -                                 - -
  ada0p1         0:99  466G linux-data                        - -
  <FREE>         -:-   7.5K -                                 - -
ada1             0:98   30G GPT                               - -
  ada1p1         0:100 200M efi                    gpt/efiboot0 -
  ada1p2         0:101 512K freebsd-boot           gpt/gptboot0 -
  <FREE>         -:-   492K -                                 - -
  ada1p3         0:102 8.0G freebsd-swap              gpt/swap0 SWAP
  ada1p4         0:103  22G freebsd-zfs                gpt/zfs0 <ZFS>
  ada1p4.eli     0:109  22G zfs                               - -
  <FREE>         -:-   836K -                                 - -

First I looked for defective parts of the disc (ada0) with dd:

# dd if=/dev/ada0 of=/dev/null  bs=1M
476940+1 records in
476940+1 records out
500107862016 bytes transferred in 5517.928814 secs (90633257 bytes/sec)

I then proceeded to delete everything:

# gpart destroy -F ada0
ada0 destroyed


And so it is now:

# lsblk ada0
DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT
ada0 0:97 466G GPT - -
<FREE> -:- 466G - - -


What I'm trying to do is have zroot/usr/home (now 2% occupied), take care of my data like photos, videos, etc. as any /home partition on the HDD and thus free up the SSD for the root and its sub-directories. since the SSD where the system is installed has very little storage.

# df -hT
Filesystem          Type     Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default  zfs       21G    7,4G     13G    36%    /
devfs               devfs    1,0K    1,0K      0B   100%    /dev
zroot/var/log       zfs       13G    436K     13G     0%    /var/log
zroot/usr/home      zfs       13G    249M     13G     2%    /usr/home
zroot               zfs       13G     88K     13G     0%    /zroot
zroot/var/crash     zfs       13G     88K     13G     0%    /var/crash
zroot/usr/ports     zfs       13G     88K     13G     0%    /usr/ports
zroot/var/mail      zfs       13G    112K     13G     0%    /var/mail
zroot/var/audit     zfs       13G     88K     13G     0%    /var/audit
zroot/tmp           zfs       13G    6,9M     13G     0%    /tmp
zroot/var/tmp       zfs       13G     88K     13G     0%    /var/tmp
zroot/usr/src       zfs       13G     88K     13G     0%    /usr/src

# cat /etc/fstab
# Device Mountpoint FStype Options Dump Pass#
/dev/ada1p3 none swap sw 0 0



I have honestly never done this and if you help me, I would like to make it happen. Then I'd have FBSD running totally, like a single disk, and it's already running at peak performance!

Thanks as always.

 

[T-Daemon]

Proceed as follows:

gpart create -s gpt ada0
gpart add -t freebsd-zfs ada0

zfs rename  zroot/usr/home zroot/usr/home0
zpool create -O compression=lz4 -O atime=off -m /usr/home 500G ada0

( 500G is the pool name, deviated from the disk size, eventually set other name )

Check mountpoints:
df -h

cp -a /usr/home0/* /usr/home

Log in as a user, check for error messages, if there are none, proceed:

zfs destroy zroot/usr/home0

If you are planing to trust valuable data on that laptop, make yourself familiar with zfs send snapshot to external disks for backups. In addition make also backups you are used to make until now.

 

[judd]

T-Daemon

First, thank you for your time !

There were errors:

root@solo:/usr/home/solo # gpart create -s gpt ada0
gpart: geom 'ada0': File exists
root@solo:/usr/home/solo # gpart add -t freebsd-zfs ada0
gpart: table 'ada0' is corrupt: Operation not permitted
root@solo:/usr/home/solo # zfs rename  zroot/usr/home zroot/usr/home0
cannot unmount '/usr/home': Device busy
root@solo:/usr/home/solo # zpool create -O compression=lz4 -O atime=off -m /usr/home 500G ada0
mountpoint '/usr/home' exists and is not empty
use '-m' option to provide a different default
root@solo:/usr/home/solo # df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default     21G    7,4G     13G    36%    /
devfs                 1,0K    1,0K      0B   100%    /dev
zroot/tmp              13G    6,8M     13G     0%    /tmp
zroot/usr/home         13G    254M     13G     2%    /usr/home
zroot/var/crash        13G     88K     13G     0%    /var/crash
zroot/usr/src          13G     88K     13G     0%    /usr/src
zroot                  13G     88K     13G     0%    /zroot
zroot/var/audit        13G     88K     13G     0%    /var/audit
zroot/var/log          13G    440K     13G     0%    /var/log
zroot/usr/ports        13G     88K     13G     0%    /usr/ports
zroot/var/mail         13G    112K     13G     0%    /var/mail
zroot/var/tmp          13G     96K     13G     0%    /var/tmp
root@solo:/usr/home/solo # cp -a /usr/home0/* /usr/home
cp: No match.
root@solo:/usr/home/solo #

 

[judd]

Right now I'm deleting again *ada0* but this one you see with dd.

# dd if=/dev/zero of=/dev/ada0 bs=1m
dd: /dev/ada0: short write on character device
dd: /dev/ada0: end of device
476941+0 records in
476940+1 records out
500107862016 bytes transferred in 5517.884153 secs (90633991 bytes/sec)

With the dd parameter it worked:

lsblk
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:97  466G GPT                               - -
  ada0p1         0:99  466G freebsd-zfs                       - <ZFS>
ada1             0:98   30G GPT                               - -
  ada1p1         0:100 200M efi                    gpt/efiboot0 -
  ada1p2         0:101 512K freebsd-boot           gpt/gptboot0 -
  <FREE>         -:-   492K -                                 - -
  ada1p3         0:102 8.0G freebsd-swap              gpt/swap0 SWAP
  ada1p4         0:103  22G freebsd-zfs                gpt/zfs0 <ZFS>
  ada1p4.eli     0:108  22G -                                 - -
  <FREE>         -:-   836K -                                 - -

df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default     21G    7,4G     13G    36%    /
devfs                 1,0K    1,0K      0B   100%    /dev
zroot/usr/home         13G    257M     13G     2%    /usr/home  >>>>>is still here.
zroot/tmp              13G    6,7M     13G     0%    /tmp
zroot/var/tmp          13G     88K     13G     0%    /var/tmp
zroot/var/crash        13G     88K     13G     0%    /var/crash
zroot/var/audit        13G     88K     13G     0%    /var/audit
zroot/usr/ports        13G     88K     13G     0%    /usr/ports
zroot/var/log          13G    448K     13G     0%    /var/log
zroot                  13G     88K     13G     0%    /zroot
zroot/usr/src          13G     88K     13G     0%    /usr/src
zroot/var/mail         13G    112K     13G     0%    /var/mail

# fdisk ada0
******* Working on device /dev/ada0 *******
parameters extracted from in-core disklabel are:
cylinders=969021 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=969021 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 238 (0xee),(EFI GPT)
    start 1, size 976773167 (476940 Meg), flag 0
        beg: cyl 0/ head 0/ sector 2;
        end: cyl 1023/ head 255/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>

# zfs destroy zroot/usr/home0
cannot open 'zroot/usr/home0': dataset does not exist. ???


Is the fstab as it is or should a line be added ?
# cat /etc/fstab

# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ada1p3             none    swap    sw              0       0

 

[T-Daemon]

root@solo:/usr/home/solo # gpart create -s gpt ada0
gpart: geom 'ada0': File exists
root@solo:/usr/home/solo # gpart add -t freebsd-zfs ada0
gpart: table 'ada0' is corrupt: Operation not permitted
root@solo:/usr/home/solo # zfs rename zroot/usr/home zroot/usr/home0
cannot unmount '/usr/home': Device busy
root@solo:/usr/home/solo # zpool create -O compression=lz4 -O atime=off -m /usr/home 500G ada0
mountpoint '/usr/home' exists and is not empty
use '-m' option to provide a different default

I expected after your gpart destroy -F ada0 ada0 with no partition table, also gpart: table 'ada0' is corrupt: Operation not permitted needs to be investigated.

Furthermore I expected you where executing those commands from /root not /usr/home/solo, this can't work from there, zroot/usr/home can't be renamed as long you are in that directory, as consequence no pool with mountpoint /usr/home on ada0 can be created when there is a second one on zroot .

On top of that I overlooked you have full disk encryption, sorry for that, it was late when I responded to your posting.

There are two options, first we could keep trying to create on ada0 a encrypted/unencrypted /usr/home, or second option install a encrypted root-on-ZFS system on ada0, and copy the data from ada1 over ( zfs send / receive system configuration, packages and their configuration and databases, etc. ).

The second option is cleaner, no scattered file system over two disks, makes in case of a system failure a full disk restoration easier.
I will be gone for a day or two, don't be alarmed when not receiving a response. In that time others can join in to help.

 

[judd]

Furthermore I expected you where executing those commands from /root not /usr/home/solo, this can't work from there, zroot/usr/home can't be renamed as long you are in that directory, as consequence no pool with mountpoint /usr/home on ada0 can be created when there is a second one on zroot .

I understand, now I understand my failure. Of course I didn't leave the directory.

I have no problem removing an installation and reinstalling so I can have /usr/home on ada0.
Although it is true, as you say, that /usr/home is not on another disk in case something needs to be fixed.

 

[Mjölnir]

• It is essential that you give distinctive names to the two zpool(8)s.
• You can use PEFS pefs-kmod and only encrypt sensible data instead of GELI encryption. pefs(8) is very mature & works on top of any filesystem. Then you can easily switch to ZFS native encryption later, once OpenZFS has stabilized (around FreeBSD 13.3 or 14.x).
• Setting vfs.zfs.mdcomp_disable="1" in loader.conf(5) will speed up file operations on the SSD.
• Since a laptop has an UPS built-in (the battery), you can optionally set vfs.zfs.cache_flush_disable=1 & vfs.zfs.vdev.bio_flush_disable=1 in sysctl.conf(5). This is non-standard -- do at your own risk -- but I never had any bad experience. Do not do this if your HDD is connected external (e.g. via USB), because if it gets disconnected accidently, the filesystem can get corrupted & result in irrecoverable data loss.
• Set zfs set atime=off on both pools and zfs inherit that for all datasets, and only have it on for /var/mail.
• Concerning overwiting the partition table see this post & the one I posted before that.

 

[judd]

[QUOTE = "mjollnir, publicación: 478153, miembro: 37644"]
Es esencial que asigne nombres distintivos a los dos zpool (8) s.
[/CITAR]

Lo escribo y luego lo leo. : +1:

[QUOTE = "mjollnir, publicación: 478153, miembro: 37644"]
You can use PEFS pefs-kmod and only encrypt sensible data instead of GELI encryption. pefs(8) is very mature & works on top of any filesystem. Then you can easily switch to ZFS native encryption later, once OpenZFS has stabilized (around FreeBSD 13.3 or 14.x).
[/QUOTE]

Ok, I have not yet experienced PEFS ...
Another thing to read, I'm running out of eyes ...

Setting vfs.zfs.mdcomp_disable="1" in loader.conf(5) will speed up file operations on the SSD.

Well, I deleted the SSD and installed FBSD on the HDD.

lsblk
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:96  466G GPT                               - -
  ada0p1         0:98  200M efi                    gpt/efiboot0 -
  ada0p2         0:99  512K freebsd-boot           gpt/gptboot0 -
  <FREE>         -:-   492K -                                 - -
  ada0p3         0:100 8.0G freebsd-swap              gpt/swap0 SWAP
  ada0p4         0:101 458G freebsd-zfs                gpt/zfs0 <ZFS>
  ada0p4.eli     0:106 458G -                                 - -
  <FREE>         -:-   4.0K -                                 - -
ada1             0:97     - -                                 - -

I'll write it down and leave it for the next installation on the SSD.

Since a laptop has an UPS built-in (the battery), you can optionally set vfs.zfs.cache_flush_disable=1 & vfs.zfs.vdev.bio_flush_disable=1 in sysctl.conf(5). This is non-standard -- do at your own risk -- but I never had any bad experience. Do not do this if your HDD is connected external (e.g. via USB), because if it gets disconnected accidently, the filesystem can get corrupted & result in irrecoverable data loss.

I proceed and then comment.
I never use an external hard drive in the laptop, I use it in the desktop computer.

Set zfs set atime=off on both pools and zfs inherit that for all datasets, and only have it on for /var/mail.

I don't understand it.

Concerning overwiting the partition table see this post & the one I posted before that.

If I have read them, they are interesting!

~~

Actually, my head and my eyes are about to explode, so I'm going to take it one step at a time. It's too much reading, although necessary, of course.