Solved Binary upgrade from 10.3-RELEASE-p11 to 11.0-RELEASE fails


After doing binary upgrade from 10.3-RELEASE-p11 to 11.0-RELEASE my system fails to boot.
I'm using the standard
#freebsd-update -r 11.0-RELEASE upgrade

It seems I'm hitting the errata notice published in FreeBSD-EN-16:18.loader
I'm using ufs partition with 512-byte sectors, my system fails to mount /root at

The only option is to reboot it with kernel.old back to 10.3-RELEASE-p11
I've noticed the following interesting fact: It's stated just before the list of 45000+ files to be updated that the system is going to be upgraded to 11.0-RELEASE-p2. However after the first execution of
#freebsd-update install
and before
#shutdown -r now
I was able to identify the newly installed kernel version to be 11.0-RELEASE-p1, affected by the FreeBSD-EN-16:18.loader errata.
I've identified this by executing
#strings /boot/kernel/kernel | grep RELEASE

I'm wondering whether the binary upgrade process is still to be fixed in order to install 11.0-RELEASE-p2 kernel directly.

Please advise!
Could you divulge some more details about your setup?
1. What hard drive arrangement (single drive, RAID type, etc.) / partitioning scheme do you use?
2. What hardware do you have?
3. Etc.?
Knowing more always helps :).
My setup uses a single SATA hard drive (Western Digital 160GB).
Please find the partition info and dmesg outputs below:

$ df -k
Filesystem  1024-blocks     Used    Avail Capacity  Mounted on
/dev/ad4s1a      507630   198176   268844    42%    /
devfs                 1        1        0   100%    /dev
/dev/ad4s1e      507630     1006   466014     0%    /tmp
/dev/ad4s1f   143322018 43989914 87866344    33%    /usr
/dev/ad4s1d     3017358   738014  2037956    27%    /var
fdescfs               1        1        0   100%    /dev/fd

$ cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ad4s1b             none            swap    sw              0       0
/dev/ad4s1a             /               ufs     rw              1       1
/dev/ad4s1e             /tmp            ufs     rw              2       2
/dev/ad4s1f             /usr            ufs     rw              2       2
/dev/ad4s1d             /var            ufs     rw              2       2
/dev/acd0               /cdrom          cd9660  ro,noauto       0       0
fdesc                   /dev/fd         fdescfs rw              0       0

$ dmesg
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.3-RELEASE-p11 #0: Mon Oct 24 18:47:18 UTC 2016 i386
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Atom(TM) CPU  330   @ 1.60GHz (1596.15-MHz 686-class CPU)
  Origin="GenuineIntel"  Id=0x106c2  Family=0x6  Model=0x1c  Stepping=2
  AMD Features=0x20100000<NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 2147483648 (2048 MB)
avail memory = 2071175168 (1975 MB)
Event timer "LAPIC" quality 400
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP/HT): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP/HT): APIC ID:  3
random: <Software, Yarrow> initialized
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <INTEL D945GLF2> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
acpi_button0: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: Length mismatch for 3 range: 6c00000 vs 6bfffff
pcib0: Length mismatch for 3 range: 70000001 vs 70000000
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0x30e0-0x30e7 mem 0x90300000-0x9037ffff,0x80000000-0x8fffffff,0x90380000-0x903bffff irq 16 at device 2.0 on pci0
agp0: <Intel 82945G (945G GMCH) SVGA controller> on vgapci0
agp0: aperture size is 256M, detected 7932k stolen memory
vgapci0: Boot video device
hdac0: <Intel 82801G HDA Controller> mem 0x903c0000-0x903c3fff irq 22 at device 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x2000-0x20ff mem 0x90200000-0x90200fff,0x90000000-0x9000ffff irq 16 at device 0.0 on pci1
re0: Using 1 MSI-X message
re0: Chip rev. 0x3c000000
re0: MAC rev. 0x00400000
miibus0: <MII bus> on re0
rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: Using defaults for TSO: 65518/35/2048
re0: Ethernet address: 00:1c:c0:bf:8a:4e
pcib2: <ACPI PCI-PCI bridge> at device 28.2 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 28.3 on pci0
pci3: <ACPI PCI bus> on pcib3
uhci0: <Intel 82801G (ICH7) USB controller USB-A> port 0x3080-0x309f irq 23 at device 29.0 on pci0
uhci0: LegSup = 0x2f00
usbus0 on uhci0
uhci1: <Intel 82801G (ICH7) USB controller USB-B> port 0x3060-0x307f irq 19 at device 29.1 on pci0
uhci1: LegSup = 0x2f00
usbus1 on uhci1
uhci2: <Intel 82801G (ICH7) USB controller USB-C> port 0x3040-0x305f irq 18 at device 29.2 on pci0
uhci2: LegSup = 0x2f00
usbus2 on uhci2
uhci3: <Intel 82801G (ICH7) USB controller USB-D> port 0x3020-0x303f irq 16 at device 29.3 on pci0
uhci3: LegSup = 0x2f00
usbus3 on uhci3
ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem 0x903c4000-0x903c43ff irq 23 at device 29.7 on pci0
usbus4: EHCI version 1.0
usbus4 on ehci0
pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci4: <ACPI PCI bus> on pcib4
fxp0: <Intel 82550C Pro/100 Ethernet> port 0x1000-0x103f mem 0x90120000-0x90120fff,0x90100000-0x9011ffff irq 21 at device 0.0 on pci4
miibus1: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> PHY 1 on miibus1
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow
fxp0: Using defaults for TSO: 65518/35/2048
fxp0: Ethernet address: 00:02:b3:e8:aa:85
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30b0-0x30bf irq 18 at device 31.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
atapci1: <Intel ICH7 SATA300 controller> port 0x30c8-0x30cf,0x30ec-0x30ef,0x30c0-0x30c7,0x30e8-0x30eb,0x30a0-0x30af irq 19 at device 31.2 on pci0
ata2: <ATA channel> at channel 0 on atapci1
ata3: <ATA channel> at channel 1 on atapci1
ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 on acpi0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xcb000-0xcbfff,0xcc000-0xcd7ff pnpid ORM0000 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
random: unblocking device.
usbus0: 12Mbps Full Speed USB v1.0
Timecounters tick every 1.000 msec
hdacc0: <Realtek ALC662 HDA CODEC> at cad 2 on hdac0
hdaa0: <Realtek ALC662 Audio Function Group> at nid 1 on hdacc0
pcm0: <Realtek ALC662 (Analog)> at nid 20 and 24,25,26 on hdaa0
pcm1: <Realtek ALC662 (Front Analog Headphones)> at nid 27 on hdaa0
pcm2: <Realtek ALC662 (Onboard Digital)> at nid 30 on hdaa0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ugen1.1: <Intel> at usbus1
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen0.1: <Intel> at usbus0
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen3.1: <Intel> at usbus3
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen2.1: <Intel> at usbus2
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
ada0 at ata2 bus 0 scbus1 target 0 lun 0
ada0: <WDC WD1600AAJS-22L7A0 01.03E01> ATA8-ACS SATA 2.x device
ada0: Serial Number WD-WMAV31922880
ada0: 150.000MB/s transfers (SATA, UDMA5, PIO 8192bytes)
ada0: 152627MB (312581808 512 byte sectors)
ada0: Previously was known as ad4
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
Timecounter "TSC" frequency 1596148620 Hz quality 1000
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/ad4s1a [rw]...
re0: link state changed to DOWN
fxp0: link state changed to DOWN
re0: link state changed to UP
pflog0: promiscuous mode enabled
Partitioning and /etc/fstab look clean. What error message pops up when you mean "my system fails to boot"? Finally, are you booting in legacy BIOS (CSM) mode or UEFI mode?
I noticed that dmesg mentions that the processor is an Intel Atom. Not sure whether this might a problem or not.
I'm using the legacy BIOS mode.

Apologies, can't paste the output from the not fully booted 11.0-RELEASE system, the last output where it freezes is something like:
Trying to mount root from /dev/ad4s1a

After pressing any key it goes to a mountroot prompt where an attempt to manually boot fails again. Please note again I'm not pasting the actual output here, just manually writing it by memory:
mountroot>ufs:/dev/ad4s1a rw
Just would like to add: Intel Atom 330 works fine with 10.3-RELEASE-p11, although I haven't checked whether this CPU might not be supported in 11.0-RELEASE. I honestly doubt this might be the case. My suspicion falls at 512-byte sectors' size on my HDD and the related FreeBSD-EN-16:18.loader errata.
I'm wondering about the device name (ad4 vs, ada0) and how/when the ad4 legacy name is set up during the boot process.

from ada(4)

This variable determines whether /dev/adY symbolic links are created,
which tries to mimic old ata(4) numbering. Set to 1 to enable legacy
aliases symlinks, 0 to disable. The default is currently enabled.

I would try to boot using the native device name, ada0s1a ...
Thanks ASX,
This machine has been initially installed with 10.0-RELEASE, then sequentially binary upgraded to 10.3-RELEASE (now with p11). No special boot device handling manipulation has been ever made.
I'd like to better understand - how should I try to boot using ada0 ?
Should I manually edit /etc/fstab
by replacing ad4 with ada0 and try to reboot the existing 10.3-RELEASE-p11 ?
Isn't there a risk of no rollback? What my options after possible failure would be?
After pressing any key it goes to a mountroot prompt where an attempt to manually boot fails again. Please note again I'm not pasting the actual output here, just manually writing it by memory:
I guess you can use that prompt to check/set the device to boot from:
lsdev should provide you a list of bootable devices.
At the prompt you could try something like:
mountroot>ufs:/dev/ada0s1a rw
and no, there are no risks, because it is however a temporary setup.

After booting, I guess that the symbolic links to ad4 will be in place, and no fstab changes are required.

If booting from ada0 will work you will need to change some loader config file to make it permanent, I'm not 100% sure which one, because I'm using grub to boot, possibly loader.conf
The compatiblity links are only a minor annoyance if you're using UFS, with ZFS they may change the device names shown in the pool and that might cause some problems.

This is to confirm
mountroot>ufs:/dev/ada0s1a rw
was successful. After this I've edited
and replaced all ad4 entries with ada0 and rebooted in 10.3-RELEASE-p11
The binary upgrade to 11.0-RELEASE was applied successfully.
It appears 11.0-RELEASE can't work with the legacy /dev/ad4
No ad4 symlinks are created pointing to ada0

Thank you all! The issue is fixed.