FreeBSD 11 amd64 memstick fails

wayne47

wayne47

I have 3 USB sticks, each of which was written with
dd if=<memstick.img> of=/dev/da0 bs=64k conv=sync
from the sources
FreeBSD-9.3-RELEASE-amd64-memstick.img
FreeBSD-10.3-RELEASE-amd64-memstick.img
FreeBSD-11.0-RELEASE-amd64-memstick.img

All of the sha512 sums validate including doing a dd back from the USB drive. I have a couple of similar older servers (8GB ECC RAM) and have tried this on both of these with equivalent results. I also tried different USB drives.

9.3 and 10.3 boot fine but 11.0 throws the error shown in this picture:
Fbsd11USB.8x6.jpg

I skimmed the forums; did not see this issue addressed. One clue: I booted a working 11.0 system with the USB drive installed (but not selected for boot) and saw this on the console:
GEOM: da0: the secondary GPT header is not in the last LBA.
One of these systems currently has the hard drives removed so running custom anything to diagnose things would not be an issue.
 
wayne47 said:
GEOM: da0: the secondary GPT header is not in the last LBA.
Click to expand...
I guess that is normal, your stick is larger then the image, otherwise it would not fit!

The actual error is the bus enumeration taking place
Code: 
INQUIRY. CDB: 12 00 00 00 24 00
The kernel is trying to find the device on the bus; no actual data read going on yet.
This is also mentioned in https://lists.freebsd.org/pipermail/freebsd-stable/2015-April/082288.html
where work-around was reported that waiting a couple of seconds to stabalize buggy xHCI would help.
So type in ufs:/dev/ufs/FreeBSD_Install after some time at the mountroot> prompt.

Another work-around might be to use mfsbsd image (http://mfsbsd.vx.sk/files/images/11/mfsbsd-se-11.0-RELEASE-amd64.img) which can at least get 11.0 kernel started to diagnose the problem further.
 
I gave it 10 minutes, tried the mountroot, still got an error.
There is something wrong about the FreeBSD 11 image though - the BIOS takes forever to recognize it as a USB device (and sometimes does not see it as a viable boot device). No such problem with 9.3 or 10.3, BIOS sees those right away and offers to boot from them.

I'll try mfsbsd
 
No luck. mfsbsd does not crash, it hangs at
GEOM: da0: the secondary GPT header is not in the last LBA.
Looks like a regression in the USB code on 11.
 
wayne47 said:
and sometimes does not see it as a viable boot device
Click to expand...
"sometimes" is bad. Something flaky in the hardware/BIOS. Do you have front/back usb to choose from? Does the BIOS have "Legacy USB" option on? For that matter, what BIOS/mobo are we talking about.
 
I just checked. Problem persists in 12 Current.

Here's the motherboard: https://www.asus.com/us/Commercial-Servers-Workstations/P5MT/

9.3 and 10.3 work fine. Every single boot. For 11, I usually "trick" the BIOS by booting 10, then choosing USB boot, rebooting and swapping the USB drive otherwise the BIOS does not think the USB drive is a boot device. Not sure on Legacy USB status. Rear USB only. Note that a DVD boot will install and run fine as well, problem is only USB. Also, the Memtest usb stick will always work as well.
 
AFAIK the bios setup on P5MT has "legacy USB" enable option on by default under bios setup Advanced->USBconfiguration
Best NOT to have your bios try to mimick things for legacy OS so you might try:
Legacy USB Support -> disabled -- I think 11.0 is not a "Legacy OS"
EHCI Handoff -> disabled -- Freebsd has ehci driver
 
It was set to Auto. Disabling Legacy USB Support also disabled EHCI Handoff.
No change in behavior.
 
Just checked - FreeBSD-10.3-RELEASE-amd64-mini-memstick.img indeed has a different setup
as compared to FreeBSD-11.0-RELEASE-amd64-mini-memstick.img
10.3 was partitioned as MBR while 11.0 is partitioned as GPT. Indeed the 11.0 image is more flexible (combined bios/uefi) - BTW. it can also be dd'ed to an ide/sata harddisk and boot just fine off an ide controller instead of via USB.

As a workaround, it is possible (I've checked it) to move the 11.0 install data to the 10.3 memstick keeping the MBR layout.
suppose da11 has 11.0 and da10 has 10.03:
Code: 
# gpart resize -i 1 da10     # grow the ufs partition to the size of the stick
# growfs /dev/da10a          # grow the filesystem itself to span the partition
# mount /dev/da10a /mnt      # mount the 10.3 ufs filesystem
# cd /mnt ; dump 0fs - 200000 /dev/da11p3 | restore -rf -      # copy over the 11.0 data
# umount /mnt
The dump restore cycle gives a lot of errors about existing symlinks, but they don't not seem to hurt when booting the reworked 10.3->11 memstick.
Are you willing to try this? It might shed some light on the situation.
 
I'll give that a try. In the interim, I have managed to get 11 to boot. I believe that the key was telling the BIOS that the USB should not be auto detected, rather force it to Hard Drive. But I was trying a lot of things.
 
OK, have some more info.
All 3 memory sets were tested with Memtest overnight (because I would point to bad RAM after reading the following). I now have 3 USB drives (11.0, 10.3, 11.0/10.3 blend from above). Also, results are repeatable on multiple servers.
  1. I pulled the CMOS battery so that I would be forced to reset to defaults on each power cycle.
  2. With 4 GB of generic non-ECC RAM, simply inserting any one of the three drives causes the system to boot from the USB drive and come up fine. System boots in a few seconds.
  3. Changed to 8GB of Kingston ECC RAM (also tested with 8GB ECC A-Tech). System now takes ~90 seconds to boot (perhaps checking ECC?). Although all three cause the BIOS to say that the USB drive is "Found and configured", the system refuses to see the USB drive as a boot device. Going into the BIOS, I can force the USB drive to be the boot device. This was likely the cause of my earlier confusion (the "sometimes") as I was using multiple servers and not noting ECC/non ECC.
  4. 10.3 and the 11.0/10.3 blend both work, 11.0 gives the mountroot error.
  5. I forced the USB to emulate a hard drive, this caused the 11.0/10.3 blend to fail at mountroot, 11.0 still fails.
  6. I then disabled USB legacy mode (leaving the Forced hard drive) and the stock 11.0 now boots.
So ECC seems to be highly relevant and USB detection is fragile. Hope the above helps.
 
Or it could be the amount of RAM... Do you have 8 GB of non-ECC RAM (or 4 GB ECC RAM) for this machine?
Still, my bet is on the bios / uefi firmware - the vendor (or whoever wrote that firmware) took some shortcuts, and now you are having trouble for it.
 
Nobody asked yet, so: are you using the latest BIOS?
Also, I notice that this is a quite an old motherboard, the latest BIOS is from 2007...
 
thanks for the information, I have hit same problem, I use easy2boot so I can have multiple os on one stick, and the author specifically states the non uefi versions of the memstick images to be used, I noticed 11 only has one version, and it seems the problem is the GPT layout instead of MBR.

I think its pretty unlikely the FreeBSD developers can backtrack on their decision so I will follow the steps in post 9
 
You must log in or register to reply here.
Back
Top