Solved Is there a problem with /var, /usr on a pool different form a / pool?

M

mefizto

Greetings all,

after trying to install /var, /usr on a different drive and pool than /, I concluded that I do not have the skill to do so. Thus, I decided to install on rpool on a first drive, create system pool on a second drive, use net/rsync to transfer the /var, /usr from the rpool to the system, and delete the /var, /usr from the rpool.

Since upon boot, the pools have to be mounted, I wonder if such a setup will not cause a problem. Consider, that during boot, there is need to write to either /var or the /usr and the system is not yet mounted.

Can such a scenario even happen, or am I worried needlessly? I cannot find any reference to the pool mounting on boot.

Kindest regards,

M
 
From my experience from having /var on a seperate pool; Yes you can get into a situation of the system pool tries to mount before rpool and fails. Simply put, it is an inherit race condition on which one that goes first. You can still do that setup, but I suggest you set the system pool to NOT auto mount and just mount those parts in your fstab. By doing it this way, it clears the race condition in that even if system pool comes up first, it will wait; so rpool will mount first, then fstab kicks in and mounts the sets from the system pool.

The key parts you will need to read up on is canmount setting to off/noauto and/or mountpoint to legacy.

As far as removing the /var and /usr directory from rpool; you can keep them their as when the system pool is mounted, it will redirect everything to the correct pool/set automatically once everything is setup properly.
 
Hi ct85711,

thank your for the reply.

Simply put, it is an inherit race condition on which one that goes first.
Click to expand...

I have gotten a different perspective from reading uefi(8), which recites in part:
boot1.efi searches partitions of type freebsd-ufs and
freebsd-zfs for loader.efi. The search begins with partitions
on the device from which boot1.efi was loaded, and continues
with other available partitions. If both freebsd-ufs and
freebsd-zfs partitions exist on the same device the
freebsd-zfs partition is preferred. boot1.efi then loads and
executes loader.efi.
Click to expand...
thus implying that the root pool is mounted first. But yes, the race condition is what I was worried, thank you for confirming.

Regarding your suggestion to mount etc/fstab, I did not know that was possible. I will read upon that. Just so I understand, the potential write to /var will be postponed until all the pools are mounted?

Regarding the redirecting, I had it experimentally set in a manner that the directories from the system were mounted over the corresponding directories at the root.

Kindest regards,

M
 
Last edited:
Yes uefi will try mounting the root pool, but it does NOT always identify which pool it is correctly when multiple are available to pick from; I am guessing it may order them by name, but that is purely guessing. Using fstab generally not necessary with zfs, since it will automatically mount the pool; but when you have multiple that need to be done in specific order; it's easier to just go old school and tell the system the order to do it. By the nature of fstab, it is executed early on in the boot process, shortly after the root filesystem is mounted; so the pool with /var gets mounted one step after the root pool is mounted.

mefizto said:
Regarding the redirecting, I had it experimentally set in a manner that the directories from the system were mounted over the corresponding directories at the root.
Click to expand...
Yes, this is the basic foundation of how stuff are mounted in bsd and linux systems, in that you mount over an existing point and that point now goes to the new mount, and any old stuff prior are effectively hidden and unused..
 
Hi ct85711,

thank your for the education re /etc/fstab. I think that in view of the recitation form uefi(8) and the fact that the pools are on different drives the rpool always be found and mounted first. However, as the race condition still may exist, i.e., the need to write into /var before the system is mounted, I will experiment with the /etc/fstab. Do you think it would be sufficient to add only the system pool or both pools?

Due to the "mounting over" and the small size of the /var, /usr upon initial installation, it may not be necessary to delete these directories form the rpool.

Kindest regards,

M
 
Just to be clear, UEFI isn't a factor in which pool is mounted first. When the computer boots up, the order the devices and drives are presented to the operating system are not ordered in any way. The easiest way of thinking this, is more of imagine you toss a dozen balls in the air; when you catch them, how do you know which order they are in? This was frequently the cause of problems with multiple nics/drives/etc and how you referenced them. In response to do, like drives, the standard changed in that you started referencing them by a unique value. This could very well be a label, or on Linux part-UUID/UUID, or some other method.

mefizto said:
the need to write into /var before the system is mounted,
Click to expand...

You are getting confused on this; in that the need to write is much later after stuff is mounted. In a basic fashion of how the process goes; the bios passes onto the boot loader. The boot loader loads the kernel (often stored on the boot partition). The kernel mounts the root partition, looks up what else to mount from fstab. After that, it starts loading the various services. In case for ZFS, the around the boot loader/kernel area, it looks at the various drives it knows of at that time and auto mounts the ZFS drives it can. The kernel it self doesn't need to write to anywhere, as it is all in memory.

So you know, you do still need to keep the base folder (/usr or /var, etc..) as the system needs that to have a location to mount to. The only 2 that are an exception is the /boot(or the /EFI) partition (if it is a seperate partition) and the '/ ' drive/partition, and the swap partition. Swap is special in that it doesn't have a fixed location, and is added to the memory address space; the base root partition since it is the base of the tree and the boot/efi are flagged for the system so it can find and access it.
 
Hi ct85711,

you are absolutely correct in that I did not understand the boot process and the sequence of events, and was, therefore, needlessly worried. Thank you for the clear explanation.

However, I am not quite sure whether I agree with the undetermined order of pool mounting, at least as it applies to my scenario. As I read the recitation:
The search begins with partitions
on the device from which boot1.efi was loaded
Click to expand...
since the rpool is on the device from which the loader was loaded, it will be found first and thus mounted first. Am I misrepresenting?

Kindest regards,

M
 
Just because it starts there, does not mean it stops after the first valid item; that is merely a starting point only and continues looking for all pools before it mounts everything. Like on my server (has 2 pools), the EFI/boot/root partition is on one drive and my /home and /var (secondary pool)) is on a separate pool/drive. Numerous times, I had where my secondary pool try mounting first (and failed) then my primary pool mounted after. System still booted, but anything that needed the contents from /var or my /home directories failed since they didn't have anything.
 
Hi ct85711,

thank you for sharing your experience. So it appears that the /etc/fstab is a good idea to prevent the scenario that you described.

On a slightly different note, I am trying the same setup like yours, i.e., to have /var, /usr, /home on a different drive and pool than /. Were you able to make the installer create such a layout or did you use a script? Any advice/help in this regards would be greatly appreciated.

Kindest regards,

M
 
As far as creating the setup, you will have to create the layout/pools yourself. You could possibly do it in the manual setup portion; I more of just let it create the first pool and install onto that pool. At the end of the installer, it gives you an option to go to the terminal for any additional setup. From there, you go ahead and setup the second pool and datasets on the other drive; and move the files over (of course updating the fstab too). The key parts to remember, is to mount the second pool's datasets using the altroot to a temporary location while also setting the pool to not auto mount. The other key part, is DO NOT perform the move on a live system (hence why you do it while you booted from the installer image).

Also, you may want to first try only moving the /var and /home directories first and boot to make sure everything mounts/works correctly; then boot the installer again and transfer the /usr directory. The reasoning is that /var and /home are not critical to having the system to start up (you'll have to login as root, not as an user).
 
Hi ct85711,

thank you for the explanation and especially for the warnings about mounting the second pool and not moving the files on a live system.

Regarding the move, how do you do that? Can I use net/rsync?

Kindest regards,

M
 
I simply just had the root pool mounted in one spot, (often I like doing it at /tmp/root) and the secondary pool's datasets in a different directories in /tmp. After that, I just did a simple mv since you aren't really moving anything to a different system. Though, I'm an noob on using rsync so doing anything with that is often a more involved affair just to use that tool.

Obviously, if you don't create any users from the installer, you don't have to worry about anything in the /home or /usr/home.

If you are doing this on a pre-existing system, you'd be safer to do a copy instead of move and make sure everything copied over correctly before going back and removing the old copies.
 
Hi ct85711,

thank you for the clarification. I was thinking about net/rsync because it can be instructed to preserve all the different attributes, e.g., links (both hard and soft), ownership, etc. But, thinking about it, it may not be available in the bootonly. I will check whether mv does the same.

It will be done on a new installation.

Kindest regards,

M
 
You must log in or register to reply here.
Back
Top