Solved FreeBSD 13 does not find the logical slices inside slice #4 of the MBR root disk

Esteemed Colleagues:

FreeBSD 13 is installed on /dev/ada0s3, which is a "primary partition" of the MBR-partitioned root disk (more specifically, the root filesystem is on /dev/ada0s3a, which begins 16 blocks away from the beginning of /dev/ada0s3 because that's where bsdlabel offered to put it, during the installation procedure, and I also have a /dev/ada0s3b which is available for swap if I ever need a swap area, which currently I am doing without).

Why did I choose MBR partitioning for my disk? Because this computer is a multiboot machine, and I intend to install operating systems on it, like ReactOS, which, to the best of my knowledge, do not know how to boot themselves from a GPT slice.

FreeBSD does not know how to install itself onto, or boot itself from, a logical slice, so I installed in onto one of the 3 primary slices. And it's a good thing that I did, because I am running it now, and it does not even see the logical slices. Behold:
Code:
# ls -las /dev/ada0s*
0 crw-r-----  1 root  operator  0x5c Nov  8 21:40 /dev/ada0s1
0 crw-r-----  1 root  operator  0x5d Nov  8 21:40 /dev/ada0s2
0 crw-r-----  1 root  operator  0x5e Nov  8 21:40 /dev/ada0s3
0 crw-r-----  1 root  operator  0x62 Nov  8 21:40 /dev/ada0s3a
0 crw-r-----  1 root  operator  0x63 Nov  8 21:40 /dev/ada0s3b
0 crw-r-----  1 root  operator  0x5f Nov  8 21:40 /dev/ada0s4
#
There should be a /dev/ada0s5 and a /dev/ada0s6, and there is not. /dev/ada0s4 is my extended slice, and I have logical slices inside of it. One is a Linux volume group containing Linux logical volumes on which various Linux distributions will be installed (at the moment, only ZorinOS is installed, but there will be others). The other I am using as a ZFS pool. It is a minor inconvenience that the FreeBSD system cannot see the Linux volume group, because I want the Linux filesystems to be visible to the FreeBSD system, which has partly functional lvm drivers and fully functional ext2 drivers. But it is a major inconvenience that the FreeBSD system cannot see the ZFS pool. The ZFS pool is where I am putting the shared storage, such as the /home filesystem, that will be common to the Linux and FreeBSD systems, as well as my not-yet-installed OpenIndiana and NetBSD systems (parenthetically, why did we have to wait years for NetBSD to get ZFS drivers that weren't broken? But I digress). If I cannot get my FreeBSD system to see my ZFS pool, then I will not use it. And this is after already investing the mandatory post-installation week that you have to spend building the ports that render a FreeBSD system minimally functional.

When my FreeBSD system boots, it issues, inter alia, the following complaints:
Code:
# dmesg | grep -i geom
GEOM_PART: integrity check failed (ada0s4, EBR)
GEOM_PART: integrity check failed (diskid/DISK-WD-WX51A3805XTFs4, EBR)
GEOM_PART: integrity check failed (diskid/DISK-WD-WX51A3805XTFs4, EBR)
GEOM_PART: integrity check failed (diskid/DISK-WD-WX51A3805XTFs4, EBR)
#
I do not know why it is complaining of a failed integrity check. My ZorinOS system has no problem finding the logical slices inside the extended slice (and a good thing too, since otherwise it could not boot, because that's where it lives). More to the point, I am doing everything I can to disable the integrity check. Behold:
Code:
# sysctl -a | grep integrity
kern.geom.part.check_integrity: 0
# cat /etc/sysctl.conf
# $FreeBSD$
#
# This file is read when going to multi-user and its contents piped thru
# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
kern.geom.part.check_integrity=0
#

It may be that I have the right idea, that
Code:
kern.geom.part.check_integrity=0
is the way to solve this problem, but that by the time /etc/sysctl.conf is read, it is too late, devfs has already been created. I tried to mount -F devfs devfs /mnt/dev after the system was up, and the command failed with the error message
Code:
mount: devfs: No such file or directory
; but I did succeed by putting the appropriate line inside /etc/fstab and doing a mount -a. It did not make a difference. There is still no /dev/ada0s5 or /dev/ada0s6 underneath /mnt/dev, just as there is none underneath /dev.

Perhaps the solution is to do the
Code:
kern.geom.part.check_integrity=0
at an earlier stage in the boot process. But I cannot figure out how to do that. One thinks of using
Code:
set
statements in the grub menu, after doing
Code:
kfreebsd /boot/kernel/kernel
, or perhaps even after doing
Code:
kfreebsd /boot/loader
. But that is not how I boot my FreeBSD system. I boot it with
Code:
set root=(hd0,3)
chainloader +1
That is the only way I have been able to boot my system. I have not been able to get any other technique to work. I cannot boot my system with
Code:
kfreebsd
, because I cannot get grub to see any filesystem on
Code:
(hd0,3)
, even after doing
Code:
insmod ufs
. Now, this might be because, as I mentioned earlier, /dev/ada0s3a begins 16 blocks after /dev/ada0s3. Behold:
Code:
# bsdlabel /dev/ada0s3
# /dev/ada0s3:
8 partitions:
#          size     offset    fstype   [fsize bsize bps/cpg]
  a:   40000000         16    unused        0     0
  b:    1943024   40000016    unused        0     0
  c:   41943040          0    unused        0     0     # "raw" part, don't edit
#
But even if that were the problem, I should be able to get grub to see the bsd subpartitioning after doing an
Code:
insmod bsd
, and I should be able to get the following to work:
Code:
insmod bsd
set root=(hd0,3,a)
insmod ufs
kfreebsd /boot/loader
But it does not work; grub chokes on the
Code:
set root=(hd0,3,a)
, because it fails to see the bsd subpartitioning.

My theory is that I might solve my problem if I can get the kernel to do
Code:
kern.geom.part.check_integrity=0
early enough, before creating the devfs filesystem and mounting it on /dev. How do I do this on a system which is booted with
Code:
chainloader +1
? Is there a file somewhere in the /boot directory where I can put that line, because it clearly doesn't work when I put it in /etc/sysctl.conf? Or am I totally wrong, and the solution lies elsewhere? Thank you in advance for any and all replies.

Jay F. Shachter
 
So let me get this straight, you put a bsdlabel in MBR position 3, an extended partition in MBR position 4, and linux-somethings in positions 1 & 2. GEOM thinks your EBR on MBR entry 4 is hosed, and you think delaying checking it will solve your problems? But you ALSO have a problem with booting FreeBSD because grub won't read the disklabel and boot the kernel directly and using boot1 that's installed on the disklabel is... something you don't want?

I had to recompile the kernel in order to change GEOM defaults at boot time.
 
The Linux installations (so far there is only one), are not in primary slices 1 or 2; they are inside a Linux volume group which is one of several "logical" slices inside of MBR 4. But that is an unimportant correction, except that it demonstrates that there is nothing wrong with the EBR. The Linux installation is able to find /dev/sda5 and /dev/sda6 (which in FreeBSD would be called /dev/ada0s5 and /dev/ada0s6). So the inability of the FreeBSD system to find them, is inexplicable.

As for booting FreeBSD, I have no problem with the "chainloader +1" technique. The only reason I mentioned using the "kfreebsd" technique, which I cannot get to work, is that the "kfreebsd" technique enables grub to put variables into the kernel environment with the "set" command. These variables are available to user applications after the system boots thru the kenv command, and, more important, they can affect the behavior of the kernel at an early stage of booting, earlier than variables that are placed in /etc/sysctl.conf. Since posting my original posting, I have learned that these kernel variables can also be mentioned in /boot/loader.conf. So now my question is: Is there a variable assignment I can place into /boot/loader.conf that has the same effect as sysctl -w kern.geom.part.check_integrity=0? If not, how else does one get FreeBSD to see my logical slices? As always, thank you in advance for any and all replies.

Jay F. Shachter
 
Just because Linux can deal with the EBR doesn't mean there's nothing wrong with it. Linux could be more forgiving about the EBR than FreeBSD. How big is the drive and is it 512e or AF?

"The variables which can be set using sysctl are named the same as the tunables in /boot/loader.conf."

Stuff "kern.geom.part.check_integrity=0" into loader.conf and find out
 
Just because Linux can deal with the EBR doesn't mean there's nothing wrong with it. Linux could be more forgiving about the EBR than FreeBSD. How big is the drive and is it 512e or AF?

"The variables which can be set using sysctl are named the same as the tunables in /boot/loader.conf."

Stuff "kern.geom.part.check_integrity=0" into loader.conf and find out
its listed by sysctl -aT so it should work
 
I thank all of the contributors to the thread for their replies. Putting kern.geom.part.check_integrity=0 into /boot/loader.conf, and then rebooting, solved my problem, and I shall mark this thread as "Solved". I still get the exact same error messages as before when the system boots, but now they are not fatal; and after boot, I now have /dev/ada0s5 and /dev/ada0s6 and /dev/ada0s7 (although they are implemented as symbolic links). Behold:
Code:
# ls -l /dev/ada0s*
crw-r-----  1 root  operator  0x5c Nov  9 17:46 /dev/ada0s1
crw-r-----  1 root  operator  0x5d Nov  9 17:46 /dev/ada0s2
crw-r-----  1 root  operator  0x5e Nov  9 17:46 /dev/ada0s3
crw-r-----  1 root  operator  0x62 Nov  9 17:47 /dev/ada0s3a
crw-r-----  1 root  operator  0x63 Nov  9 17:47 /dev/ada0s3b
crw-r-----  1 root  operator  0x5f Nov  9 17:46 /dev/ada0s4
crw-r-----  1 root  operator  0x64 Nov  9 17:46 /dev/ada0s4+00000001
crw-r-----  1 root  operator  0x66 Nov  9 17:46 /dev/ada0s4+00000033
crw-r-----  1 root  operator  0x68 Nov  9 17:46 /dev/ada0s4+06657686
lrwxr-xr-x  1 root  wheel       15 Nov  9 17:46 /dev/ada0s5 -> ada0s4+00000001
lrwxr-xr-x  1 root  wheel       15 Nov  9 17:46 /dev/ada0s6 -> ada0s4+00000033
lrwxr-xr-x  1 root  wheel       15 Nov  9 17:46 /dev/ada0s7 -> ada0s4+06657686
#

I am therefore now able to access my zfs pool, and I am able to access my Linux volume group, which, as you will soon see, both reside in logical partitions inside of the disk's extended slice, /dev/ada0s4. Behold:

Code:
# zpool import -f vendikar
# zpool status
  pool: vendikar
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
    still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
    pool will no longer be accessible on software that does not support
    feature flags.
config:

    NAME               STATE     READ WRITE CKSUM
    vendikar           ONLINE       0     0     0
      ada0s4+00000033  ONLINE       0     0     0

errors: No known data errors
#
# geom linux_lvm load
# geom linux_lvm list
Geom name: vendikar
Providers:
1. Name: linux_lvm/vendikar-zorin
   Mediasize: 19327352832 (18G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: ada0s4+06657686
   Mediasize: 75161927680 (70G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

#
# mount -t ext2fs /dev/linux_lvm/vendikar-zorin /ZorinOS
#

So, at this point, I am almost satisfied with FreeBSD13. All I need to do now is get the system to recognize the external VGA monitor, and then it will be usable. But that is a topic for another posting, in another forum.

Many thanks again to all who participated in this thread.

Jay F. Shachter
 
I boot rEFInd boot manager on BSD partition with the following on an external 128GB usb flash drive. With GPT partitioning
set root='(hd1,gpt2)'
ls (hd1,gpt1)/efi/boot
chainloader (hd1,gpt1)/efi/boot/BOOTX64-REFIND.EFI
boot

gpart show
=> 34 468877245 ada0 GPT (224G)
34 4062 - free - (2.0M)
4096 614400 1 efi (300M)
618496 449799648 2 linux-data (214G)
450418144 7 - free - (3.5K)
450418151 18454920 3 linux-swap (8.8G)
468873071 4208 - free - (2.1M)

=> 34 468877245 diskid/DISK-191078801079 GPT (224G)
34 4062 - free - (2.0M)
4096 614400 1 efi (300M)
618496 449799648 2 linux-data (214G)
450418144 7 - free - (3.5K)
450418151 18454920 3 linux-swap (8.8G)
468873071 4208 - free - (2.1M)

=> 40 250626488 da0 GPT (120G)
40 532480 1 efi (260M)
532520 232595456 2 freebsd-zfs (111G)
233127976 17487872 3 freebsd-swap (8.3G)
250615848 10680 - free - (5.2M)

camcontrol devlist
<SanDisk SSD PLUS 240GB UF5000RL> at scbus0 target 0 lun 0 (pass0,ada0)
<PLDS DVD-RW DS8A8SH KU51> at scbus1 target 0 lun 0 (cd0,pass1)
<AHCI SGPIO Enclosure 2.00 0001> at scbus3 target 0 lun 0 (pass2,ses0)
<Samsung Flash Drive 1100> at scbus4 target 0 lun 0 (da0,pass3)

gpart status
Name Status Components
ada0p1 OK ada0
ada0p2 OK ada0
ada0p3 OK ada0
diskid/DISK-191078801079p1 OK diskid/DISK-191078801079
diskid/DISK-191078801079p2 OK diskid/DISK-191078801079
diskid/DISK-191078801079p3 OK diskid/DISK-191078801079
da0p1 OK da0
da0p2 OK da0
da0p3 OK da0


geom disk list
Geom name: ada0
Providers:
1. Name: ada0
Mediasize: 240065183744 (224G)
Sectorsize: 512
Mode: r0w0e0
descr: SanDisk SSD PLUS 240GB
lunid: 5001b448b8428296
ident: 191078801079
rotationrate: 0
fwsectors: 63
fwheads: 16

Geom name: cd0
Providers:
1. Name: cd0
Mediasize: 0 (0B)
Sectorsize: 2048
Mode: r0w0e0
descr: PLDS DVD-RW DS8A8SH
ident: (null)
rotationrate: unknown
fwsectors: 0
fwheads: 0

Geom name: da0
Providers:
1. Name: da0
Mediasize: 128320801792 (120G)
Sectorsize: 512
Mode: r2w2e4
descr: Samsung Flash Drive
lunid: 534d490033323830
ident: 0375720100004849
rotationrate: unknown
fwsectors: 63
fwheads: 255
 
What does Chainloader +1 mean?


Therefore +1 actually means: The block list starting at offset 0 with length 1 on GRUB's root device. This simply means the first block, which is the MBR of the root device. ... However, in this case BIOS loads GRUB, which in turn loads the code in MBR, thus forming a chain. This is why it's called a chainloader .Dec 27, 2016
Where is the stage 1 of GRUB usually located?


MBR

Stage 1 is the piece of GRUB that resides in the MBR or the boot sector of another partition or drive. Since the main portion of GRUB is too large to fit into the 512 bytes of a boot sector, Stage 1 is used to transfer control to the next stage, either Stage 1.5 or Stage 2.
What is a Chainloader?


Chainloading is when a boot loader loads another boot loader to begin the boot process.Apr 14, 2016
https://blog.cykerway.com/posts/2016/12/27/grub-chainloader-1.html Explanation of Chainloader +1
 
Back
Top