Solved Booting from RAID-Z on FreeBSD 10.1-RC1 fails

C

cheffobg

Hi,

I have an HP Microserver Gen8 and have some problems running FreeBSD 10.1* on it. The install goes all fine, but once I have to boot the new system the attached loader panic happens.
file.php


If I switch the storage controller from BIOS to Legacy (from AHCI) the system boots fine, but... no Command queuing and etc.

Here is a snippet from dmesg:
Code: 
atapci0: <Intel Cougar Point SATA300 controller> port 0x10c0-0x10c7,0x10c8-0x10cb,0x10d0-0x10d7,0x10d8-0x10db,0x10e0-0x10ef,0x10f0-0x10ff irq 17 at device 31.2 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
atapci1: <Intel Cougar Point SATA300 controller> port 0x1400-0x1407,0x1408-0x140b,0x1410-0x1417,0x1418-0x141b,0x1420-0x142f,0x1430-0x143f irq 17 at device 31.5 on pci0
ata4: <ATA channel> at channel 0 on atapci1
ata5: <ATA channel> at channel 1 on atapci1

ada0 at ata2 bus 0 scbus0 target 0 lun 0
ada0: <ST3000VN000-1H4167 SC43> ATA-9 SATA 3.x device
ada0: Serial Number Z3101P71
ada0: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes)
ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ata2 bus 0 scbus0 target 1 lun 0
ada1: <ST3000VN000-1H4167 SC43> ATA-9 SATA 3.x device
ada1: Serial Number Z3101X6H
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad5
ada2 at ata3 bus 0 scbus1 target 0 lun 0
ada2: <ST3000VN000-1H4167 SC43> ATA-9 SATA 3.x device
ada2: Serial Number Z3101PBF
ada2: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes)
ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad6
ada3 at ata3 bus 0 scbus1 target 1 lun 0
ada3: <ST3000VN000-1H4167 SC43> ATA-9 SATA 3.x device
ada3: Serial Number Z3101VWD
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada3: Previously was known as ad7

gpart show ada0 (This is valid for all four disks)
=>        34  5860533101  ada0  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40         512     1  freebsd-boot  (256K)
         552  5860532576     2  freebsd-zfs  [bootme]  (2.7T)
  5860533128           7        - free -  (3.5K)

zpool status
  pool: zroot
 state: ONLINE
  scan: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        zroot          ONLINE       0     0     0
          raidz1-0     ONLINE       0     0     0
            gpt/disk0  ONLINE       0     0     0
            gpt/disk1  ONLINE       0     0     0
            gpt/disk2  ONLINE       0     0     0
            gpt/disk3  ONLINE       0     0     0

errors: No known data errors
I have tried booting from a/the ZFS mirror but still no luck. I believe the problem is somewhere between gptzfsloader, ahci, loader and the BIOS but can't figure it out. I've installed the system using as a base this guide - https://wiki.freebsd.org/RootOnZFS/GPTZ ... .0-RELEASE and also tried the auto ZFS install (RAID-Z and mirror) - it all goes OK until I have to reboot.

Here is a dmesg with AHCI on and booting from the install USB memstick:
Code: 
ahci0: <Intel Cougar Point AHCI SATA controller> port 0x10c0-0x10c7,0x10c8-0x10cb,0x10d0-0x10d7,0x10d8-0x10db,0x10e0-0x10ff mem 0xfacd0000-0xfacd07ff irq 17 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
-- snip --
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ST3000VN000-1H4167 SC43> ATA-9 SATA 3.x device
ada0: Serial Number Z3101P71
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled

ahci0@pci0:0:31:2:	class=0x010601 card=0x330d103c chip=0x1c028086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '6 Series/C200 Series Chipset Family SATA AHCI Controller'
    class      = mass storage
    subclass   = SATA
All BIOS settings are factory defaults. Only Soft RAID is changed to Legacy/AHCI. Any ideas what is broken and how to fix it?

P.S. Just so Google can index, the error is:
Code: 
Can't workout which disk we are booting from. 
Guessed BIOS device 0xffffffff not found by probes, defaulting to disk0:
file softcore.c, line 428
 

Attachments

  • Capture.PNG
    Capture.PNG
    15.8 KB · Views: 687
Last edited by a moderator:
The 10.1-RC1 announcement describes the following:

http://lists.freebsd.org/pipermail/freebsd-stable/2014-October/080453.html
Important note to ZFS users on the i386 architecture: A regression has been discovered that affects multi-disk (mirror, raidz-1, raidz-2, etc.) installations that may cause a kernel panic on boot. If using a multi-disk ZFS setup, adding 'options KSTACK_PAGES=4' is suspected to resolve the problem. Please *do* *not* upgrade your system with freebsd-update(8) if using a multi-disk ZFS setup, since this will override the kernel configuration with the GENERIC kernel.
Click to expand...

I would suggesting asking or following the FreeBSD-STABLE mailing list for any of the latest discussions on a fix. Hopefully that should be fixed soon.
 
Last edited by a moderator:
The described regression doesn't seem to be related to this problem. It looks more like a BIOS bug/configuration problem to me because the error says "can't work out which disk we are booting from" and that comes from the boot loader before the kernel is loaded and started. I would poke around the BIOS and try different settings of boot priority to see if that makes the difference.
 
Unfortunately the BIOS does not allow to "poke" with disk priority. I can choose from USB, CD, floppy, network and HDD (C:).

Also, if you look at the screenshot you will notice that the same message is displayed twice i.e. the gpzfsloader is executing itself again which is probably the reason to not see the HDDs properly.
 
I run into the same problem with FreeBSD-10.1-RELEASE-amd64. I installed the system onto a pair of USB flash drives with HDD's detached, it boots up fine. When I attach the HDD's without a partition table, it still boots up fine, once I partitioned the drive with GPT, it stopped booting with the same error message as yours.

FreeNAS-9.3-BETA works fine, it uses GRUB and loads the kernel directly. The issue looks most likely cause by zfsloader to me.
 
kuriyama@ said:
Is it possible to test with https://people.freebsd.org/~kuriyama/hp-bios/20141209/zfsloader?

This is generated with a patch from here: https://github.com/kuriyama/freebsd/compare/freebsd:stable/10...hp-bios-workaround-10.diff

I also have the same problem (DL320e v2 + 4KB sector size HDD), and tracking this down. I can boot with this zfsloader.

Of course you cannot boot this box now, you need boot from CD-ROM etc. and mount the target HDD and install zfsloader on it.
Click to expand...
kuriyama@ you ROCK!
I wish I saw this earlier. I was trying to build a home NAS with the MicroServer Gen8, and decided to give the newly released FreeNAS 9.3 a try. Lots of data has been imported into the box and so far I am happy about it. Will be absolutely happy to test the zfsloader by you if I changed my mind to use vanilla FreeBSD instead of FreeNAS. And will be happy to see your patch get committed. Thanks again.
 
Hi,

I ran into the same situation today - fresh install of 10.1-RELEASE on a HP MicroServer Gen8, SATA-Ports set to AHCP, ZRAID-1, and zfsloader crash with the same error message. Using the zfsloader binary from kuriyama@ resolves the problem, server boots nicely.

Thanks!

gert
 
It was probably clear enough to you guys, but it took me some struggling to figure out the process. I thought I'd write it up, here, to help the next guy. I was upgrading a server by replacing the old motherboard with the motherboard from a ProLiant ML10v2. I had 8 disks attached to an LSI card flashed to IT mode, four disks attached to the B120i via a SATA fanout cable, and one boot disk connected to one of the two available SATA ports.

What worked for me was:
• switch the boot disk to the first B120i port (on the SATA fanout cable), giving me the boot disk and 3 SATA disk on the fanout cable, and one more SATA disk on one of the motherboard's SATA ports
• reboot the server and hit <F9> to enter setup
• in the "System Options" menu, use the "SATA Controller Options" item to switch the "Embedded SATA Configuration" from RAID to "Legacy"
• boot the server using the boot disk on port1
• use wget to fetch the zfsloader binary, above
• in my /boot directory move the zfsloader binary to zfsloader.old
• copy the zfsloader binary I had downloaded to the /boot directory
• reinstalled the ZFS boot loader ( gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0)
• rebooted the server and used <F9> to go back into setup
• used "System Options > SATA Controller Options" to switch the configuration from "Legacy" to "AHCI"

Now I'm able to boot, and hot swap works. I have 13 disks, total, all working as expected. Thank you all!
 
Hi,
I have the same problem with HP Microserver Gen8 and Freebsd 11.
I have Freebsd installed on a SSD sata connected to the internal USB port, 2 x 4 Tb hard drive on the first two bays.
When the SATA controller is in AHCI mode, I have this error

Code: 
Can't work out which disk we are booting from.
Guessed BIOS device 0xffffffff not found by probes, defaulting to disk0: allocat not found
panic: Assertion failed: (FALSE), function ficlCompileSoftCore, file softcore line 428.

If I set the SATA controller in Legacy mode, I can boot.
I found the wiki page with a bootloader for 10.1 and 10.3 but not for 11.0. Can I use the same 10.3 bootloader?
 
You must log in or register to reply here.
Back
Top