Installer hangs during boot (11.1 on AMD)

Rafal Lukawiecki

Rafal Lukawiecki

I am unable to install/run FreeBSD 11.1 (or FreeNAS 11.0) on my brand-new HPE MicroServer Gen10 containing an AMD Opteron X3421 with 16GB RAM (see specs). When installing either FreeBSD or FreeNAS the installer hangs just after it prints:

pci0: <ACPI PCI bus> on pcib0

Boot.jpg

And with verbose boot on:

Verbose boot.jpg

I have spent some six hours trying many alternative BIOS options: with UEFI and with legacy boot, with/without RAM bank interleaving, with/without RAM channel interleaving, and with forcing a specific memory clock as well as leaving that on auto, with/without CPU CPB, C6, and STAPM mode options, with/without PCI above 4G decoding mode, with/without UEFI networking, with/without USB XHCI and EHCI handoff options.

I have also tried the various FreeBSD boot options. Safe mode had no impact. Disabling ACPI caused:

panic: running without device atpic requires a local APIC

No ACPI Panic.jpg

I have also tried installing using different USB sticks and I have checked the integrity of the installer images. I have also tried moving an already-installed FreeNAS/FreeBSD to this machine, and it also hangs at exactly the same place during the boot.

I am able to run Linux on it, having succeeded with CentOS 7 and ArchLinux. As I have run out of options and I am somewhat new to FreeBSD (but not Unix) I would greatly appreciate any suggestions how to troubleshoot this set-up—will be glad to try anything! This server was purchased specifically for FreeBSD/FreeNAS, as an upgrade from an older HP MicroServer Gen7 on which it run well (but slowly).

Apologies for the literal screen-shots above, but this was off a VGA feed from the server.

Edit. It looks like others are having similar issues, see this thread on FreeNAS forums from yesterday. So far, there are no suggestions how to solve it: https://forums.freenas.org/index.ph...r-a-few-seconds-on-a-microserver-gen10.56809/
 
k.jacker said:
Hei Rafal,
Try to boot with ACPI disabled. Vom the boot screen (with ASCII beasty) hit '6' and then toogle 'ACPI support' to off with '3'.
Greetings
Matthias
Click to expand...
Thanks for suggesting this, Matthias. I have already tried that. This causes panic: running without device atpic requires a local APIC. See my 3rd screenshot above for the messages displayed when I try without ACPI.
 
Me again. I went through some manpages for event timers and releated as from your screenshots it look like trouble with hpet and related. Without beeing an expert with kernel and interupt stuff I'd like to push you in the right direction and maybe you could try something out.
Read the manpages for apic(4), attimer(4) and hpet(4) and see if the device hints (hint.X.X....) from the manpages help you work around your problem.
It works like this:
Boot, hit '3' to drop to the loader prompt and then use the set command. F. eks. set hint.apic.0.disabled=1 to disable apic and use legacy pic.
You possibly need set those for every CPU in your system (hint.apic.0.disabled=1 - hint.apic.2.disabled=1 and so on).

When your done, try your settings by typing boot and hit enter. That's what I would do, even if there is no guaranty anything of those device hints will help. Worth a try, though :)

Greetings
Matthias
 
jef said:
As a long shot, there is suggestion that the Marvell 88SE9230 controller may have issues with non-Windows systems. That chip is a "new" to the HPE ProLiant Gen10 MicroServer according to https://homeservershow.com/forums/t...10-microserver-beta-to-previous-microservers/
Click to expand...
Oh dear. I have just re-read that thread, and I am not sure if it is conclusive that this system is unable to run FreeBSD or that it causes issues with ZFS (which I was planning to continue using when I have migrated).

I will try suggestions by k.jacker to see if I can get past the initial boot problems.

On a side-note, I wonder how often one comes across a major hardware manufacturer's piece of current tech that is point-blank incompatible with FreeBSD.
 
k.jacker said:
Me again. I went through some manpages for event timers and releated as from your screenshots it look like trouble with hpet and related. Without beeing an expert with kernel and interupt stuff I'd like to push you in the right direction and maybe you could try something out.
Read the manpages for apic(4), attimer(4) and hpet(4) and see if the device hints (hint.X.X....) from the manpages help you work around your problem.
It works like this:
Boot, hit '3' to drop to the loader prompt and then use the set command. F. eks. set hint.apic.0.disabled=1 to disable apic and use legacy pic.
You possibly need set those for every CPU in your system (hint.apic.0.disabled=1 - hint.apic.2.disabled=1 and so on).

When your done, try your settings by typing boot and hit enter. That's what I would do, even if there is no guaranty anything of those device hints will help. Worth a try, though :)

Greetings
Matthias
Click to expand...
Dear Matthias, I have just tried your suggestion. I have entered four sets: set hint.apic.0.disabled=1 till set hint.apic.3.disabled=1, then boot. I got the following panic this time:
APIC Disabled Panic.jpg
On another note—please bear in mind that I do not understand the kernel's boot process or its needs—it seems to me that the verbose boot output suggested that the hang occurred after the ACPI stuff has already been initialised. To re-quote my earlier screenshot:
Verbose boot.jpg
...one can see that the hang happens when (or just after) the boot process deals with the memory on pcib0. I apologise if I do not understand how to interpret this correctly.

I am now worried that I may have to send this brand-new HPE MicroServer Gen10 back to the retailer if I cannot troubleshoot something as basic as boot... :(
 
Hei Rafal,

sad to hear it's not working.
From my experience with hardware/motherboards and FreeBSD it's mostly one manufacturer's motherboard or a BIOS revision of a motherboard that does not work (or not work well, most often APCI problems because of crappy BIOS code). While simply a newer BIOS revision or just changing the motherboard (while keeping all other hardware) makes everything work perfect.
So sometimes updating the BIOS/UEFI or just waiting for a new revision has fixed problems for me, but the last years I used to just buy another motherboard when such problems arose. Of course that's sadly not working with your HPE.

But as jef mentioned, it could simply be the Marvell chip not beening supported yet.
If you haven't come across the hardware compatibility page for your release, it's allways nice to check here if things you plan on buying are supported.
https://www.freebsd.org/releases/11.1R/hardware.html

Greetings
Matthias
 
k.jacker said:
If you haven't come across the hardware compatibility page for your release, it's allways nice to check here if things you plan on buying are supported.
https://www.freebsd.org/releases/11.1R/hardware.html
Click to expand...

One thing to be very careful about is that certain large PC vendors have, at least in the past, installed "custom" chips that have the same identifiers to the OS, but lack functionality of the generally available parts. Dell was notorious about this years ago. It was a problem with any OS, FreeBSD, Linux-based, and even Windows, which required Dell-specific drivers.
 
I can add that, on advice of a freebsd-stable mail list member, I have re-run my tests on a number of FreeBSD releases ranging from 9.3 to 12.0-CURRENT. Unfortunately, the issue affects all of them. For the avoidance of doubt, these are the releases I have just tested:

FreeBSD-9.3-RELEASE-amd64-memstick.img
FreeBSD-11.1-RELEASE-amd64-memstick.img
FreeBSD-11.1-STABLE-amd64-20170807-r322164-memstick.img
FreeBSD-12.0-CURRENT-amd64-20170807-r322167-memstick.img
 
Rafal Lukawiecki
One thing I found useful to trouble shoot boot hang issues, is to figure out what comes after the last console message. That sometimes is the thing that is causing the hang. That said, you might have to try loading FreeBSD on another box, even a simple desk top and see what comes after the hung message. Unfortunantely, I don't have a server I can try this on, which has remote console access. (So I can capture and paste the output here.) Last, different hardware may act differently, so your mileage may vary.
 
Thank you, Bob Bishop, for the patch. I am impressed that you and the community were able to fix this, but I am sorry I will not be able to test it, as I have since returned the Gen 10 MicroServer. It has been replaced with a very-well functioning SuperMicro machine, which runs FreeBSD like a charm.
 
I had the same issue when I tried to install FreeBSD 11.2 RELEASE AMD64 on an older server I retired (it's going to get a nice second life at home :)). This server has a TYAN S7045 motherboard (S7045GM4NR) and was running a V1.01 BIOS. After updating the BIOS to V1.06, the issue was resolved and the server booted fine.

The odd thing is that this server has been running FreeBSD for years. And I can't imagine that older versions of FreeBSD (9.X at the time) didn't have this issue. This means that I somehow managed to workaround this issue. But I can't remember how... My best guess is that I hooked a DVD-drive to a SATA-port and installed FreeBSD that way. :-/
 
You must log in or register to reply here.
Back
Top