Root on ZFS = slow boot?

D

Dies_Irae

Hi all,

A few months ago (see this) I upgraded my home PC from 9.2-STABLE + UFS to 10.0-STABLE + ZFS. Everything works fine, but I still have the "slow boot" problem, and I'm unable to find useful information on the net.

The "slow speed" is visible during the third stage of the boot process. The "\|/-" characters rotation is slightly slower than usual (with "usual", I mean every other FreeBSD version I've used in the past), but the real difference is visible during the kernel and modules loading process.

Loading the kernel and a few modules (like nvidia and zfs, 3 or 4 in total) took about 30 seconds.

After that, the speed seems to increment a bit, but the whole boot process took about 90 seconds. The first 60-70 seconds are barely sufficient to reach the
Code: 
Trying to mount root from...
line, and during the kernel boot process I don't see any hangs or timeouts. The kernel keeps loading at a constant speed. Once the root filesystem is mounted, everything else (starting services) seems to work fine (and fast as expected).

I've done a test by installing a minimal system on a blank disk using a 10-RELEASE install CD:
  • using UFS, the system boots in less than 30 seconds
  • using ZFS, the system boots in about 60 seconds

The slowdown again is visible mainly in the kernel + modules loading phase. Using UFS, kernel and modules are loaded almost instantly.

As far as I know, the disk partition(s) are correctly aligned:
Code: 
=>       34  156301421  ada0  GPT  (75G)
         34          6        - free -  (3.0K)
         40       1024     1  freebsd-boot  (512K)
       1064        984        - free -  (492K)
       2048  156299264     2  freebsd-zfs  (75G)
  156301312        143        - free -  (72K)

The problem seems to be related to the filesystem access (reading /boot/kernel/kernel from ZFS is ~10 times slower than from UFS).

Any idea?
 
Small update: during my daily battle with this problem, I tried to install FreeBSD on an old noisy PATA disk just to have an "audio feedback" of what's happening during the boot process (yes, I know this method is not very professional, but I'm really running out of ideas). During a sequential read (for example using dd(1)) the disk doesn't make much noise, because the head movements are very limited. During a random read, you can clearly hear the head movements.

So my thought was: /boot/kernel/kernel should not be fragmented (specially on a freshly installed system), so loading this file should be straight and quite silent, right?

Wrong. Kernel and modules loading is noisy (and slow) like a fully random disk read.

I don't really know what else to try. I've tried several disks, partitons were always correctly aligned, I've used -RELEASE and -STABLE installations, clones of my actual system and freshly installed ones, upgrade the pool to the latest version/features and who knows what else.
Nothing. Same result. Slow like booting from the CD.

The only remaining test I can think of is physically move the disk to another machine with a different motherboard and see what will happen. My actual motherboard lacks AHCI support, the BIOS only have IDE/RAID options (I'm using IDE mode, I know that using ZFS on top of hardware RAID is BAD) . Unless ZFS is higly dependent on AHCI, I don't think this could be a problem, because as I said in the previous post the same system installed using UFS is blazingly fast.

In the meantime I'll keep trying... (maybe just to discover, in the end, that ZFS boot speed is slower, and nothing else :e )
 
My guess is that on your system the BIOS routines that have to be used by the ZFS bootcode and loader happen to be really slow. On my rather recent mini ITX system booting from ZFS is maybe a tad slower than it was on UFS. Something like five seconds slower to get to the kernel start from the POST screen. Booting from UFS was always rather fast considering the system is a lowly ATOM D510.
 
You must log in or register to reply here.
Back
Top