Solved I lost bootcode in BSD slice, a lot. Help me found out WHY?

[Jaskie]

Hi guys. I am a newbie in the BSD world. I've been using Linux for a few year just as desktop stuff, no development-related work. A few days ago I decided to give FreeBSD with ZFS a try. Anyway, I installed FreeBSD dual booting Windows in a disk, the laptop is legacy BIOS boot with MBR partition scheme.

Some system info:

adam@freebsd: ~ @21:17 % gpart show -p ada0
=>        63  1953525105    ada0  MBR  (932G)
          63        1985          - free -  (993K)
        2048   268207168  ada0s1  ntfs  (128G)
   268209216         960          - free -  (480K)
   268210176     1169408  ada0s2  !39  (571M)
   269379584   629145600  ada0s3  ntfs  (300G)
   898525184  1054999984  ada0s4  freebsd  [active]  (503G)

adam@freebsd: ~ @21:17 % gpart show -p ada0s4
=>         0  1054999984   ada0s4  BSD  (503G)
           0     6291456  ada0s4a  freebsd-swap  (3.0G)
     6291456  1048708528  ada0s4b  freebsd-zfs  (500G)

adam@freebsd: ~ @21:18 % freebsd-version -kru                                                                                                  
13.0-RELEASE-p4                                                                                                                                
13.0-RELEASE-p4                                                                                                                                
13.0-RELEASE-p4                                                                                                                                
adam@freebsd: ~ @21:18 % uname -srm                                                                                                            
FreeBSD 13.0-RELEASE-p4 amd64

adam@freebsd: ~ @21:20 % cat /etc/rc.conf
zfs_enable="YES"
hostname="freebsd.asus"
wlans_ath0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"
dbus_enable="YES"
lightdm_enable="YES"
kld_list="nvidia fusefs acpi_asus acpi_asus_wmi"
ifconfig_alc0="DHCP"
clear_tmp_enable="YES"
clear_tmp_X="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
dumpdev="AUTO"
rc_startmsgs="NO"
background_dhclient="YES"
adam@freebsd: ~ @21:20 % cat /boot/loader.conf
autoboot_delay="3"
zfs_load="YES"
hw.usb.no_boot_wait=1

# boot and tty resolution and font size
vbe_max_resolution="720p"
screen.font="10x20"

# quiet boot
boot_mute="YES"

hw.acpi.lid_switch_state="S3"

And ZFS filesystems:

adam@freebsd: ~ @22:01 % zpool status -v
  pool: zroot
 state: ONLINE
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          ada0s4b   ONLINE       0     0     0

errors: No known data errors

adam@freebsd: ~ @22:02 % zfs list
NAME                 USED  AVAIL     REFER  MOUNTPOINT
zroot               35.9G   449G       96K  /zroot
zroot/ROOT          3.45G   449G       96K  none
zroot/ROOT/default  3.45G   449G     3.45G  /
zroot/tmp            148K   449G      148K  /tmp
zroot/usr           32.4G   449G       96K  /usr
zroot/usr/home      32.4G   449G     32.4G  /usr/home
zroot/usr/ports       96K   449G       96K  /usr/ports

adam@freebsd: ~ @22:02 % mount
zroot/ROOT/default on / (zfs, local, nfsv4acls)
devfs on /dev (devfs)
zroot/usr/ports on /usr/ports (zfs, local, nosuid, nfsv4acls)
zroot on /zroot (zfs, local, nfsv4acls)
zroot/tmp on /tmp (zfs, local, nosuid, nfsv4acls)
zroot/usr/home on /usr/home (zfs, local, nfsv4acls)

adam@freebsd: ~ @22:02 % cat /etc/fstab
/dev/ada0s4a    none    swap    sw      0       0

It was hard to install FreeBSD in a MBR slice with ZFSOnRoot, but I managed it. FYI, this was how I did it: https://gist.github.com/JackieMium/bf8622908bd7e3765b8a7141bb493868 . I wrote it as a note so that someone could found out if there was something wrong.

I am now running FreeBSD RELEASE-13.0-p4 with Xfce4 as DE, Nvidia legacy dirver nvidia-driver-340-340.108_3, everything seems to work just fine, except one ANNOYING thing: I lost bootcode in ada0s4, a lot. Usually, when I reboot, or hard reboot as when I was trying out suspend to RAM (S3 state) and screen wouldn't wake up. The reboot usually failed and stuck at:

forever and the onlything I could do was to reboot into FreeBSD liveCD and install bootcode into ada0s4(a) again by:

dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada0s4
dd if=/boot/zfsboot of=/dev/ada0s4a skip=1 seek=1024

It has happened for more ten times over the last a few days and I really have no clue why.

Is it that FreeBSD ZFS_on_Root in a MBR slice dual booting other system not well supported? Or am I doing something wrong and messing with the boot code or kernel? Or am I experiencing this due to some hardware failure? But as I recall. this also happened in a virtual machine with I was trying in VirtualVBox too.

Please help me find out why. If you need more infomation, let me know and I'll add more. Thank you!

 

[SirDice]

The picture shows boot0cfg(8), which isn't the standard boot code that gets installed with the installer.

dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada0s4

I'm wondering what the purpose of the dd(1) is here. This just copies the first 512 bytes from it (zfsboot itself is 262656 bytes). So this gets you only a partial bootcode. Which probably explains why it's not working.

 

[Jaskie]

The picture shows boot0cfg(8), which isn't the standard boot code that gets installed with the installer.

dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada0s4

I'm wondering what the purpose of the dd(1) is here. This just copies the first 512 bytes from it (zfsboot itself is 262656 bytes). So this gets you only a partial bootcode. Which probably explains why it's not working.

Thanks for your quick reply.

I don't really understand the whole boot process due to my very limited knowledge about computer stuff. The code was copied from tutorials I followed (see links at the end of the gist I posted, especially the one from freebsd questions mail list). But I also see it in ZFSBOOT(8) example section and it was referring to MBR slice boot too. Code was almost the same. And it seems it works, and sometimes not. That I really don't understand. If it works, it shoud just work unless I changed kernel/boot code stuff. But usually I might just have changed settings in rc.conf or loader.conf, why would I lose bootcode in ada0s4(a)?

 

[SirDice]

I don't really understand the whole boot process due to my very limited knowledge about computer stuff.

Booting a computer is quite a complicated undertaking, you never notice this because you just turn it on and the system simply starts up. When you turn on a computer the BIOS does some initialization and testing of the hardware itself (the so-called POST; Power On Self Test). It will then read the fist block of the first (hard)disk (depends on the configured boot order in the BIOS), that's the boot sector or Master Boot Record (hence the MBR name). The code from that bootsector is then executed. That bootsector can only contain about 420 bytes of code, you can't really do much in that small space. So it typically has just enough logic in there to find the correct partition (or slice) to load more code from so it can do the more complicated things like reading from a filesystem. The zfsboot code you wrote to the slice knows just enough about ZFS to be able to load a file from it, that's usually loader(8). That loader(8) will do lots of interesting stuff, like loading the kernel, modules, etc. and start then starts that kernel. The kernel will initialize everything and knows how to handle disks, filesystems and whatnot.

 

[Jaskie]

When you turn on a computer the BIOS does some initialization and testing of the hardware itself (the so-called POST; Power On Self Test). It will then read the fist block of the first (hard)disk (depends on the configured boot order in the BIOS), that's the boot sector or Master Boot Record (hence the MBR name). The code from that bootsector is then executed. That bootsector can only contain about 420 bytes of code, you can't really do much in that small space. So it typically has just enough logic in there to find the correct partition (or slice) to load more code from so it can do the more complicated things like reading from a filesystem. The zfsboot code you wrote to the slice knows just enough about ZFS to be able to load a file from it, that's usually loader(8). That loader(8) will do lots of interesting stuff, like loading the kernel, modules, etc. and start then starts that kernel. The kernel will initialize everything and knows how to handle disks, filesystems and whatnot.

I read the zfsboot(8) example again, should I use

gpart bootcode -b /boot/boot0 ada0
gpart set -a active -i 1 ada0
dd if=/dev/zero of=/dev/ada0s4 count=2
dd if=/boot/zfsboot of=/dev/ada0s4 count=1
dd if=/boot/zfsboot of=/dev/ada0s4 iseek=1 oseek=1024

to reinstall boot code to ada0s4? The example does not really fits my partition scheme because I have a swap partition in ada0s4.

 

[VladiBG]

I just will point where is your error.

forever and the onlything I could do was to reboot into FreeBSD liveCD and install bootcode into ada0s4(a) again

dd if=/boot/zfsboot of=/dev/ada0s4a skip=1 seek=1024

0 6291456 ada0s4a freebsd-swap (3.0G)

Chapter 15. The FreeBSD Booting Process

An introduction to the FreeBSD Booting Process, demonstrates how to customize the FreeBSD boot process, including everything that happens until the FreeBSD kernel has started, probed for devices, and started init

docs.freebsd.org

RootOnZFS - FreeBSD Wiki

Here's step by step example of installing zfsboot into BSD slice, also note that it's installed on the freebsd-zfs after the first sector which contain the ZFS metadata.

RootOnZFS/ZFSBootPartition - FreeBSD Wiki

 

[SirDice]

The example does not really fits my partition scheme because I have a swap partition in ada0s4.

No, your swap partition is ada0s4a, ada0s4 is a slice. That naming convention is pretty confusing for new users but the distinction between a partition and a slice is important.

Chapter 3. FreeBSD Basics

Basic commands and functionality of the FreeBSD operating system

docs.freebsd.org

 

[Jaskie]

I just will point where is your error.

Chapter 15. The FreeBSD Booting Process

An introduction to the FreeBSD Booting Process, demonstrates how to customize the FreeBSD boot process, including everything that happens until the FreeBSD kernel has started, probed for devices, and started init

docs.freebsd.org

RootOnZFS - FreeBSD Wiki

I did read the wiki about RootOnZFS, but it never lead to a successful boot after installing system when I was trying in a VM. The boot usually just didn't even got into boot1 I guess? There was no choose F1 F2 screen, just missing operating system or missing disk upon fresh boot. That's why I fianlly didn't use bsdinstall at all. I followed the tutorial in the mail list answer and it worked, and it was the only one tutorial I found to work for MBR sclice boot with ZFS.

Do you mean I should run :

dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada0s4
dd if=/boot/zfsboot of=/dev/ada0s4b skip=1 seek=1024

Instead? sorry if I am not following you or misunderstanding.

No, your swap partition is ada0s4a, ada0s4 is a slice. That naming convention is pretty confusing for new users but the distinction between a partition and a slice is important.

Chapter 3. FreeBSD Basics

Basic commands and functionality of the FreeBSD operating system

docs.freebsd.org

Yes, I did notice ada0s4a was swap and I understood the difference between a slice and a partition. But there are more than one tutorials I found using the code to install zfsboot to a partition (ada0s4a not ada0s4), see the links at the end of the gist, and in the Wiki link VladiBG posted used dd if=/mnt2/boot/zfsboot of=/dev/ad0s3a skip=1 seek=1024.

 

[SirDice]

Normally freebsd-zfs is the a partition and freebsd-swap is the b partition. The instructions you followed did this the other way around. The zfsboot code has to be written to the freebsd-zfs partition, which in your case is b.

 

[VladiBG]

Your swap partition is wiped after the reboot. That's why you are unable to boot after the restart.

zfsboot contains boot1 and BTX boot2
the first 512 bytes in zfsboot is the boot1 which need to be copied into your BSD slice aka /dev/ada0s4
The rest of the zfsboot from 513->eof need to be copied after the metadata on your freebsd-zfs which is the first 1024bytes of /dev/ada0s4b If you by mistake overwrite the metadata of the pool you will lost your zfs installation.

note:
I can't see any benefit of using ZFS on such scenario. It would be much easy of using UFS.

Note2:
you can use hd to view the content of the zfsboot
hd -n 512 zfsboot | less will show you the first 512 bytes of the file
hd -s 512 zfsboot | less will show the rest of the file
Also it can be used to view the MBR or PMBR for GPT information of the disk.
hd -n 512 /dev/ada0

 

[Jaskie]

Thank you very much SirDice and VladiBG for your guildence. Should I boot into liveCD and run

dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada0s4
dd if=/boot/zfsboot of=/dev/ada0s4b skip=1 seek=1024

to fix this? Will it destroy my existing ZFS pool or the whole system? As I said, I am new to both FreeBSD and ZFS, that's probably why I am now in such a strange position where my partition scheme is not very traditional and boot process is awkward too. Thanks again. I'll try this if it's the thing to do.

 

[VladiBG]

Before writing anything into your freebsd-zfs i would recommend to make a backup of your files from that pool, just to be on the safe side.

Yes it may destroy the pool if there's metadata after the first 2 sectors (1024) on freebsd-zfs. So yes there's a risk to lose the information.

 

[Jaskie]

Before writing anything into your freebsd-zfs i would recommend to make a backup of your files from that pool, just to be on the safe side.

But the thing is, I am yet to learn how to backup and normal filesystem (ext4, UFS or NTFS) now, let alone ZFS filesystems.

 

[Jaskie]

VladiBG No, it doesn't seem to work. I am noe stuck at F1 F2 F4 screen again (see picture in I posted in original post).
I Don't know if this is happening because it shouldn't work at all or because I have destroyed zpool by installing bootcode to ada0s4b thus preventing the system from booting.

 

[VladiBG]

Most likely yes. You can try again to install zfsboot (boot2) into the swap partition to see if you can still boot from the freebsd-zfs or import the pool from within the live CD.

If you are still learning it would be much easy for you to install the FreeBSD inside the virtualization so you can test the basics there.

 

[Jaskie]

Most likely yes. You can try again to install boot2 into the swap partition to see if you can still boot from the freebsd-zfs or import the pool from within the live CD.

Do you mean revert to ada0s4a instead? I think that'll work. But I may have to try tomorrow. It's really getting late here and I have to go to the bed now. Will keep you posted tomorrow. Thank you again!

 

[Jaskie]

VladiBG I installed zfsboot to ada0s4a (swap) partition and now it's booting again. I am starting to feel that it should be in the first sector of the first partition in BSD slice, not the partition that holds the ZFS/system. I thought I have tried all this in a VM and made sure of it. The question now is still why I constantly lose bootcode.

 

[VladiBG]

The question now is still why I constantly lose bootcode.

Your swap partition is cleared on reboot.

You can try to shrink your freebsd-swap and then create a small freebsd-zfs partition holding your boot code.

adam@freebsd: ~ @21:17 % gpart show -p ada0s4
=>         0  1054999984   ada0s4  BSD  (503G)
           0     6291456  ada0s4a  freebsd-swap  (3.0G)
     6291456  1048708528  ada0s4b  freebsd-zfs  (500G)

1. Boot using live CD.
2. Delete your freebsd-swap with gpart delete -i 1 ada0s4
3. Create a freebsd-zfs partition gpart add -s 300MB -t freebsd-zfs ada0s4
4. Create a freebsd-swap partition gpart add -t freebsd-swap ada0s4
5. Write the bootcode into the new freebsd-zfs partition

dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada0s4
dd if=/boot/zfsboot of=/dev/ada0s4a skip=1 seek=1024

6. You need to edit your fstab because now your swap partition is changed from /dev/ada0s4a to /dev/ada0s4b so create a temporary mount point under tmp mkdir /tmp/root
7. import zroot pool under /tmp/root zpool import -fR /tmp/root zroot
8. mount the /etc from the zroot under /tmp/root/ zfs mount zroot/ROOT/default
9. Edit fstab with ee /tmp/root/etc/fstab and change /dev/ada0s4a to /dev/ada0s4b to reflect the new swap partition. Save the file by pressing "esc" -> "leave editor" -> "save changes"
10. unmount zfs and export the zroot

cd /
zfs unmount -a
zpool export zroot

The final BSD slice should look like this

ada0s4  BSD  (503G)
ada0s4a  freebsd-zfs (300MB)
ada0s4b  freebsd-swap  (2.7G)
ada0s4c  freebsd-zfs  (500G)

 

[Jaskie]

Your swap partition is cleared on reboot.

You can try to shrink your freebsd-swap and then create a small freebsd-zfs partition holding your boot code.

Thanks again VladiBG for your time and patience. The example you provided seemed viable. I didn't realize this could work too as I always assumed the partition has to be the one holding the system.
Someone in FreeBSD telegram group also reminded me that I installed bootcode into swap partition, which might get cleaned upon reboot and have caused the boot code missing problem. But I didn't see your reply then and I reinstalled system to let ada0s4a be ZFS root filesystem and ada0s4b be the swap. I installed other softwares and changed some parameters after installing system and did reboot several times. Now the bootcode missing problem seemed to have been resolved.
I might test your solution in a VM. Thanks again for a million times, you're really kind.

 

[Jaskie]

VladiBG So I did some tests today in a FreeBSD VM. Almost the the same partition scheme as my laptop, just to test if the swap got cleared was the reason.

Here is the disk partition layout for this VM:

adam@bsd ~ % gpart show -p ada0
=>      63  50331585    ada0  MBR  (24G)
        63   2097152  ada0s1  ntfs  (1.0G)
   2097215   4194304  ada0s2  ntfs  (2.0G)
   6291519  44040129  ada0s3  freebsd  [active]  (21G)


adam@bsd ~ % gpart show -p ada0s3
=>       0  44040129   ada0s3  BSD  (21G)
         0   2097152  ada0s3a  freebsd-swap  (1.0G)
   2097152  41942977  ada0s3b  freebsd-zfs  (20G)

adam@bsd ~ % mount
zroot/ROOT/default on / (zfs, local, nfsv4acls)
devfs on /dev (devfs)
zroot/usr/home on /usr/home (zfs, local, nfsv4acls)
zroot/tmp on /tmp (zfs, local, nosuid, nfsv4acls)
zroot/usr/ports on /usr/ports (zfs, local, nosuid, nfsv4acls)
zroot on /zroot (zfs, local, nfsv4acls)

adam@bsd ~ % swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/ada0s3a      1048576        0  1048576     0%

I then ran `swapoff -a` and `swapon -a`, reboot and all was fine, boot code still in place. So I am starting to think that my boot code in swap partition might not the problem here.

 

[VladiBG]

swapoff and spawon just turn off and on your swap they are not clearing the swap content from freebsd-swap partition. When your system get low on memory and start pushing memory pages into the swap it deletes your BTX bootcode that previously occupied your freebsd-swap partition.

Here is the example of that in virtual machine.

root@test:~ # gpart show
=>       63  104857537  da0  MBR  (50G)
         63          1       - free -  (512B)
         64  104857536    1  freebsd  [active]  (50G)

=>     1  177513  cd0  MBR  (347M)
       1  177513       - free -  (347M)

=>        0  104857536  da0s1  BSD  (50G)
          0    1048576      1  freebsd-swap  (512M)
    1048576    7340032         - free -  (3.5G)
    8388608   96468928      4  freebsd-zfs  (46G)
----
root@test:~ # hd -n 512 -s 524288 /dev/da0s1a | less
00080000  eb 0e 42 54 58 01 02 80  f6 0f b0 06 00 20 00 00  |..BTX........ ..|
00080010  fa 31 c0 8e d0 bc 00 18  8e c0 8e d8 66 6a 02 66  |.1..........fj.f|
00080020  9d bf 00 5e b9 00 19 f3  ab bb 48 95 b9 10 00 bf  |...^......H.....|
00080030  80 00 89 1d 47 47 ab 83  c3 04 e2 f6 bf 00 5e be  |....GG........^.|
00080040  02 96 ac 98 91 e3 1d ac  92 ad 93 ad b6 08 d1 eb  |................|
00080050  73 0b 89 05 88 75 02 88  55 05 83 c0 04 8d 7d 08  |s....u..U.....}.|
00080060  e2 ec eb de c6 45 05 18  c6 45 08 10 c6 45 66 68  |.....E...E...Efh|
00080070  bb 20 28 e8 c5 00 67 0f  01 1d f6 95 00 00 67 0f  |. (...g.......g.|
00080080  01 15 f0 95 00 00 0f 20  c0 40 0f 22 c0 ea 92 90  |....... .@."....|
00080090  08 00 31 c9 b1 10 8e d1  b1 38 0f 00 d9 ba 00 a0  |..1......8......|
000800a0  00 00 36 0f b7 05 13 04  00 00 c1 e0 0a 2d 00 10  |..6..........-..|
000800b0  00 00 29 d0 b1 33 51 50  68 02 02 00 00 6a 2b ff  |..)..3QPh....j+.|
000800c0  35 0c 90 00 00 51 51 51  51 52 b1 07 6a 00 e2 fc  |5....QQQQR..j...|
000800d0  61 07 1f 0f a1 0f a9 cf  fa bc 00 18 00 00 0f 20  |a.............. |
000800e0  c0 25 ff ff ff 7f 0f 22  c0 31 c9 0f 22 d9 2e 0f  |.%.....".1.."...|
000800f0  01 15 f0 95 00 00 66 ea  fc 90 18 00 b1 20 8e d1  |......f...... ..|
00080100  8e d9 8e c1 8e e1 8e e9  48 0f 22 c0 ea 11 91 00  |........H.".....|
00080110  00 31 c0 8e d0 8e d8 bb  08 70 e8 1e 00 67 0f 01  |.1.......p...g..|
00080120  1d fc 95 00 00 fb 67 f6  05 07 90 00 00 01 74 fe  |......g.......t.|
00080130  c7 06 72 04 34 12 ea f0  ff 00 f0 e4 21 50 e4 a1  |..r.4.......!P..|
00080140  50 b0 11 e6 20 e6 a0 88  d8 e6 21 88 f8 e6 a1 b0  |P... .....!.....|
00080150  04 e6 21 b0 02 e6 a1 b0  01 e6 21 e6 a1 58 e6 a1  |..!.......!..X..|
00080160  58 e6 21 c3 6a 00 eb 34  6a 01 eb 30 6a 03 eb 2c  |X.!.j..4j..0j..,|
00080170  6a 04 eb 28 6a 05 eb 24  6a 06 eb 20 6a 07 eb 1c  |j..(j..$j.. j...|
00080180  6a 08 eb 20 6a 0a eb 1c  6a 0b eb 18 6a 0c eb 14  |j.. j...j...j...|
00080190  6a 0d eb 10 6a 0e eb 0c  6a 10 eb 00 ff 34 24 c6  |j...j...j....4$.|
000801a0  44 24 04 00 fc 1e 06 60  0f a8 0f a0 1e 06 66 83  |D$.....`......f.|
000801b0  7c 24 44 08 75 03 16 eb  04 ff 74 24 50 ff 74 24  ||$D.u.....t$P.t$|
000801c0  50 6a 10 1f 1e 07 89 e3  be 21 96 00 00 bf 00 18  |Pj.......!......|
000801d0  00 00 57 e8 56 02 00 00  5e e8 0c 03 00 00 8d 64  |..W.V...^......d|
000801e0  24 18 61 07 1f 80 3c 24  03 74 15 80 3c 24 01 75  |$.a...<$.t..<$.u|
000801f0  0a f7 44 24 10 00 01 00  00 75 05 e9 d8 fe ff ff  |..D$.....u......|
00080200

root@test:~ # swapoff -a
swapoff: removing /dev/da0s1a as swap device

root@test:~ # hd -n 512 /dev/da0s1a
00000000  fc 31 c9 8e c1 8e d9 8e  d1 bc 00 7c e8 12 01 be  |.1.........|....|
00000010  ee 7d 66 31 c0 66 bb 00  80 00 00 e8 ad 00 b9 01  |.}f1.f..........|
00000020  00 be be 81 b6 01 80 7c  04 a5 75 07 e3 18 f6 04  |.......|..u.....|
00000030  80 75 13 83 c6 10 fe c6  80 fe 05 72 e9 49 e3 e1  |.u.........r.I..|
00000040  be 76 7d e9 b6 00 89 16  00 09 b9 00 02 66 b8 00  |.v}..........f..|
00000050  04 00 00 66 bb 00 90 00  00 66 60 e8 6d 00 66 61  |...f.....f`.m.fa|
00000060  66 40 66 81 c3 00 02 00  00 e2 ee bb 00 90 8b 77  |f@f............w|
00000070  0a 01 de 4e 89 f0 c1 e8  04 05 00 40 83 e6 0f 8e  |...N.......@....|
00000080  d8 b8 00 0c 05 00 40 8e  c0 bb 08 00 81 c6 00 80  |......@.........|
00000090  8c d8 2d 00 08 8e d8 bf  ff 7f 8c c0 2d 00 08 8e  |..-.........-...|
000000a0  c0 b9 00 80 fd f3 a4 4b  75 e2 8e d9 8e c1 fc fa  |.......Ku.......|
000000b0  49 74 14 e4 64 a8 02 75  f7 b0 d1 e6 64 e4 64 a8  |It..d..u....d.d.|
000000c0  02 75 fa b0 df e6 60 fb  e9 45 13 66 31 c9 66 03  |.u....`..E.f1.f.|
000000d0  44 08 66 83 d1 00 66 51  66 50 66 c1 eb 04 53 6a  |D.f...fQfPf...Sj|
000000e0  00 6a 01 6a 10 89 e6 b4  42 cd 13 72 04 8d 64 10  |.j.j....B..r..d.|
000000f0  c3 88 e0 bf 73 7d e8 49  00 be 67 7d e8 1c 00 be  |....s}.I..g}....|
00000100  81 7d e8 16 00 30 e4 cd  16 c7 06 72 04 34 12 ea  |.}...0.....r.4..|
00000110  00 00 ff ff bb 07 00 b4  0e cd 10 ac 84 c0 75 f4  |..............u.|
00000120  c3 80 fa 80 72 17 bb aa  55 52 b4 41 cd 13 5a 72  |....r...UR.A..Zr|
00000130  0c 81 fb 55 aa 75 06 f6  c1 01 74 01 c3 be 55 7d  |...U.u....t...U}|
00000140  eb ba 50 c0 e8 04 e8 01  00 58 24 0f 3c 0a 1c 69  |..P......X$.<..i|
00000150  2f 0c 20 aa c3 43 48 53  20 6e 6f 74 20 73 75 70  |/. ..CHS not sup|
00000160  70 6f 72 74 65 64 00 52  65 61 64 20 65 72 72 6f  |ported.Read erro|
00000170  72 3a 20 58 58 00 42 6f  6f 74 20 65 72 72 6f 72  |r: XX.Boot error|
00000180  00 0d 0a 00 90 90 90 90  90 90 90 90 90 90 90 90  |................|
00000190  90 90 90 90 90 90 90 90  90 90 90 90 90 90 90 90  |................|
*
000001b0  90 90 90 90 90 90 90 90  90 90 90 90 90 90 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 80 00  |................|
000001f0  01 00 a5 fe ff ff 00 00  00 00 50 c3 00 00 55 aa  |..........P...U.|
00000200

root@test:~ # hd -n 512 -s 524288 /dev/da0s1a | less

00080000  eb 0e 42 54 58 01 02 80  f6 0f b0 06 00 20 00 00  |..BTX........ ..|
00080010  fa 31 c0 8e d0 bc 00 18  8e c0 8e d8 66 6a 02 66  |.1..........fj.f|
00080020  9d bf 00 5e b9 00 19 f3  ab bb 48 95 b9 10 00 bf  |...^......H.....|
00080030  80 00 89 1d 47 47 ab 83  c3 04 e2 f6 bf 00 5e be  |....GG........^.|
00080040  02 96 ac 98 91 e3 1d ac  92 ad 93 ad b6 08 d1 eb  |................|
00080050  73 0b 89 05 88 75 02 88  55 05 83 c0 04 8d 7d 08  |s....u..U.....}.|
00080060  e2 ec eb de c6 45 05 18  c6 45 08 10 c6 45 66 68  |.....E...E...Efh|
00080070  bb 20 28 e8 c5 00 67 0f  01 1d f6 95 00 00 67 0f  |. (...g.......g.|
00080080  01 15 f0 95 00 00 0f 20  c0 40 0f 22 c0 ea 92 90  |....... .@."....|
00080090  08 00 31 c9 b1 10 8e d1  b1 38 0f 00 d9 ba 00 a0  |..1......8......|
000800a0  00 00 36 0f b7 05 13 04  00 00 c1 e0 0a 2d 00 10  |..6..........-..|
000800b0  00 00 29 d0 b1 33 51 50  68 02 02 00 00 6a 2b ff  |..)..3QPh....j+.|
000800c0  35 0c 90 00 00 51 51 51  51 52 b1 07 6a 00 e2 fc  |5....QQQQR..j...|
000800d0  61 07 1f 0f a1 0f a9 cf  fa bc 00 18 00 00 0f 20  |a.............. |
000800e0  c0 25 ff ff ff 7f 0f 22  c0 31 c9 0f 22 d9 2e 0f  |.%.....".1.."...|
000800f0  01 15 f0 95 00 00 66 ea  fc 90 18 00 b1 20 8e d1  |......f...... ..|
00080100  8e d9 8e c1 8e e1 8e e9  48 0f 22 c0 ea 11 91 00  |........H.".....|
00080110  00 31 c0 8e d0 8e d8 bb  08 70 e8 1e 00 67 0f 01  |.1.......p...g..|
00080120  1d fc 95 00 00 fb 67 f6  05 07 90 00 00 01 74 fe  |......g.......t.|
00080130  c7 06 72 04 34 12 ea f0  ff 00 f0 e4 21 50 e4 a1  |..r.4.......!P..|
00080140  50 b0 11 e6 20 e6 a0 88  d8 e6 21 88 f8 e6 a1 b0  |P... .....!.....|
00080150  04 e6 21 b0 02 e6 a1 b0  01 e6 21 e6 a1 58 e6 a1  |..!.......!..X..|
00080160  58 e6 21 c3 6a 00 eb 34  6a 01 eb 30 6a 03 eb 2c  |X.!.j..4j..0j..,|
00080170  6a 04 eb 28 6a 05 eb 24  6a 06 eb 20 6a 07 eb 1c  |j..(j..$j.. j...|
00080180  6a 08 eb 20 6a 0a eb 1c  6a 0b eb 18 6a 0c eb 14  |j.. j...j...j...|
00080190  6a 0d eb 10 6a 0e eb 0c  6a 10 eb 00 ff 34 24 c6  |j...j...j....4$.|
000801a0  44 24 04 00 fc 1e 06 60  0f a8 0f a0 1e 06 66 83  |D$.....`......f.|
000801b0  7c 24 44 08 75 03 16 eb  04 ff 74 24 50 ff 74 24  ||$D.u.....t$P.t$|
000801c0  50 6a 10 1f 1e 07 89 e3  be 21 96 00 00 bf 00 18  |Pj.......!......|
000801d0  00 00 57 e8 56 02 00 00  5e e8 0c 03 00 00 8d 64  |..W.V...^......d|
000801e0  24 18 61 07 1f 80 3c 24  03 74 15 80 3c 24 01 75  |$.a...<$.t..<$.u|
000801f0  0a f7 44 24 10 00 01 00  00 75 05 e9 d8 fe ff ff  |..D$.....u......|
00080200

root@test:~ # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/da0s1a        524288        0   524288     0%


root@test:~ # sysctl -n hw.realmem
536870912

root@test:~ # bc
>>> 536870912/1024/1024
512
>>> quit


root@test:~ # mdconfig -s 512m -u md0
root@test:~ # newfs -U /dev/md0
root@test:~ # mount /dev/md0 /mnt
root@test:~ # dd if=/dev/random of=/mnt/junk.txt
root@test:~ # umount /mnt
root@test:~ # mdconfig -du md0


root@test:~ # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/da0s1a        524288   324216   200072    62%


root@test:~ # hd -n 512 /dev/da0s1a
00000000  fc 31 c9 8e c1 8e d9 8e  d1 bc 00 7c e8 12 01 be  |.1.........|....|
00000010  ee 7d 66 31 c0 66 bb 00  80 00 00 e8 ad 00 b9 01  |.}f1.f..........|
00000020  00 be be 81 b6 01 80 7c  04 a5 75 07 e3 18 f6 04  |.......|..u.....|
00000030  80 75 13 83 c6 10 fe c6  80 fe 05 72 e9 49 e3 e1  |.u.........r.I..|
00000040  be 76 7d e9 b6 00 89 16  00 09 b9 00 02 66 b8 00  |.v}..........f..|
00000050  04 00 00 66 bb 00 90 00  00 66 60 e8 6d 00 66 61  |...f.....f`.m.fa|
00000060  66 40 66 81 c3 00 02 00  00 e2 ee bb 00 90 8b 77  |f@f............w|
00000070  0a 01 de 4e 89 f0 c1 e8  04 05 00 40 83 e6 0f 8e  |...N.......@....|
00000080  d8 b8 00 0c 05 00 40 8e  c0 bb 08 00 81 c6 00 80  |......@.........|
00000090  8c d8 2d 00 08 8e d8 bf  ff 7f 8c c0 2d 00 08 8e  |..-.........-...|
000000a0  c0 b9 00 80 fd f3 a4 4b  75 e2 8e d9 8e c1 fc fa  |.......Ku.......|
000000b0  49 74 14 e4 64 a8 02 75  f7 b0 d1 e6 64 e4 64 a8  |It..d..u....d.d.|
000000c0  02 75 fa b0 df e6 60 fb  e9 45 13 66 31 c9 66 03  |.u....`..E.f1.f.|
000000d0  44 08 66 83 d1 00 66 51  66 50 66 c1 eb 04 53 6a  |D.f...fQfPf...Sj|
000000e0  00 6a 01 6a 10 89 e6 b4  42 cd 13 72 04 8d 64 10  |.j.j....B..r..d.|
000000f0  c3 88 e0 bf 73 7d e8 49  00 be 67 7d e8 1c 00 be  |....s}.I..g}....|
00000100  81 7d e8 16 00 30 e4 cd  16 c7 06 72 04 34 12 ea  |.}...0.....r.4..|
00000110  00 00 ff ff bb 07 00 b4  0e cd 10 ac 84 c0 75 f4  |..............u.|
00000120  c3 80 fa 80 72 17 bb aa  55 52 b4 41 cd 13 5a 72  |....r...UR.A..Zr|
00000130  0c 81 fb 55 aa 75 06 f6  c1 01 74 01 c3 be 55 7d  |...U.u....t...U}|
00000140  eb ba 50 c0 e8 04 e8 01  00 58 24 0f 3c 0a 1c 69  |..P......X$.<..i|
00000150  2f 0c 20 aa c3 43 48 53  20 6e 6f 74 20 73 75 70  |/. ..CHS not sup|
00000160  70 6f 72 74 65 64 00 52  65 61 64 20 65 72 72 6f  |ported.Read erro|
00000170  72 3a 20 58 58 00 42 6f  6f 74 20 65 72 72 6f 72  |r: XX.Boot error|
00000180  00 0d 0a 00 90 90 90 90  90 90 90 90 90 90 90 90  |................|
00000190  90 90 90 90 90 90 90 90  90 90 90 90 90 90 90 90  |................|
*
000001b0  90 90 90 90 90 90 90 90  90 90 90 90 90 90 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 80 00  |................|
000001f0  01 00 a5 fe ff ff 00 00  00 00 50 c3 00 00 55 aa  |..........P...U.|
00000200

root@test:~ # hd -v -n 512 -s 524288 /dev/da0s1a | less

00080000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080100  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080120  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080200

As you see the first sector of 512 bytes doesn't get overwritten but the BTX that is located at 1024sectors * 512bytes = 524288bytes is cleared. That's why your system is unable to boot when you write zfsboot into the freebsd-swap.

 

[Jaskie]

swapoff and spawon just turn off and on your swap they are not clearing the swap content. When your system get low on memory and start pushing memory pages into the swap it deletes your BTX bootcode that previously occupied your freebsd-swap partition.

Here is the example of that in virtual machine.

root@test:~ # gpart show
=>       63  104857537  da0  MBR  (50G)
         63          1       - free -  (512B)
         64  104857536    1  freebsd  [active]  (50G)

=>     1  177513  cd0  MBR  (347M)
       1  177513       - free -  (347M)

=>        0  104857536  da0s1  BSD  (50G)
          0    1048576      1  freebsd-swap  (512M)
    1048576    7340032         - free -  (3.5G)
    8388608   96468928      4  freebsd-zfs  (46G)
----
root@test:~ # hd -n 512 -s 524288 /dev/da0s1a | less
00080000  eb 0e 42 54 58 01 02 80  f6 0f b0 06 00 20 00 00  |..BTX........ ..|
00080010  fa 31 c0 8e d0 bc 00 18  8e c0 8e d8 66 6a 02 66  |.1..........fj.f|
00080020  9d bf 00 5e b9 00 19 f3  ab bb 48 95 b9 10 00 bf  |...^......H.....|
00080030  80 00 89 1d 47 47 ab 83  c3 04 e2 f6 bf 00 5e be  |....GG........^.|
00080040  02 96 ac 98 91 e3 1d ac  92 ad 93 ad b6 08 d1 eb  |................|
00080050  73 0b 89 05 88 75 02 88  55 05 83 c0 04 8d 7d 08  |s....u..U.....}.|
00080060  e2 ec eb de c6 45 05 18  c6 45 08 10 c6 45 66 68  |.....E...E...Efh|
00080070  bb 20 28 e8 c5 00 67 0f  01 1d f6 95 00 00 67 0f  |. (...g.......g.|
00080080  01 15 f0 95 00 00 0f 20  c0 40 0f 22 c0 ea 92 90  |....... .@."....|
00080090  08 00 31 c9 b1 10 8e d1  b1 38 0f 00 d9 ba 00 a0  |..1......8......|
000800a0  00 00 36 0f b7 05 13 04  00 00 c1 e0 0a 2d 00 10  |..6..........-..|
000800b0  00 00 29 d0 b1 33 51 50  68 02 02 00 00 6a 2b ff  |..)..3QPh....j+.|
000800c0  35 0c 90 00 00 51 51 51  51 52 b1 07 6a 00 e2 fc  |5....QQQQR..j...|
000800d0  61 07 1f 0f a1 0f a9 cf  fa bc 00 18 00 00 0f 20  |a.............. |
000800e0  c0 25 ff ff ff 7f 0f 22  c0 31 c9 0f 22 d9 2e 0f  |.%.....".1.."...|
000800f0  01 15 f0 95 00 00 66 ea  fc 90 18 00 b1 20 8e d1  |......f...... ..|
00080100  8e d9 8e c1 8e e1 8e e9  48 0f 22 c0 ea 11 91 00  |........H.".....|
00080110  00 31 c0 8e d0 8e d8 bb  08 70 e8 1e 00 67 0f 01  |.1.......p...g..|
00080120  1d fc 95 00 00 fb 67 f6  05 07 90 00 00 01 74 fe  |......g.......t.|
00080130  c7 06 72 04 34 12 ea f0  ff 00 f0 e4 21 50 e4 a1  |..r.4.......!P..|
00080140  50 b0 11 e6 20 e6 a0 88  d8 e6 21 88 f8 e6 a1 b0  |P... .....!.....|
00080150  04 e6 21 b0 02 e6 a1 b0  01 e6 21 e6 a1 58 e6 a1  |..!.......!..X..|
00080160  58 e6 21 c3 6a 00 eb 34  6a 01 eb 30 6a 03 eb 2c  |X.!.j..4j..0j..,|
00080170  6a 04 eb 28 6a 05 eb 24  6a 06 eb 20 6a 07 eb 1c  |j..(j..$j.. j...|
00080180  6a 08 eb 20 6a 0a eb 1c  6a 0b eb 18 6a 0c eb 14  |j.. j...j...j...|
00080190  6a 0d eb 10 6a 0e eb 0c  6a 10 eb 00 ff 34 24 c6  |j...j...j....4$.|
000801a0  44 24 04 00 fc 1e 06 60  0f a8 0f a0 1e 06 66 83  |D$.....`......f.|
000801b0  7c 24 44 08 75 03 16 eb  04 ff 74 24 50 ff 74 24  ||$D.u.....t$P.t$|
000801c0  50 6a 10 1f 1e 07 89 e3  be 21 96 00 00 bf 00 18  |Pj.......!......|
000801d0  00 00 57 e8 56 02 00 00  5e e8 0c 03 00 00 8d 64  |..W.V...^......d|
000801e0  24 18 61 07 1f 80 3c 24  03 74 15 80 3c 24 01 75  |$.a...<$.t..<$.u|
000801f0  0a f7 44 24 10 00 01 00  00 75 05 e9 d8 fe ff ff  |..D$.....u......|
00080200

root@test:~ # swapoff -a
swapoff: removing /dev/da0s1a as swap device

root@test:~ # hd -n 512 /dev/da0s1a
00000000  fc 31 c9 8e c1 8e d9 8e  d1 bc 00 7c e8 12 01 be  |.1.........|....|
00000010  ee 7d 66 31 c0 66 bb 00  80 00 00 e8 ad 00 b9 01  |.}f1.f..........|
00000020  00 be be 81 b6 01 80 7c  04 a5 75 07 e3 18 f6 04  |.......|..u.....|
00000030  80 75 13 83 c6 10 fe c6  80 fe 05 72 e9 49 e3 e1  |.u.........r.I..|
00000040  be 76 7d e9 b6 00 89 16  00 09 b9 00 02 66 b8 00  |.v}..........f..|
00000050  04 00 00 66 bb 00 90 00  00 66 60 e8 6d 00 66 61  |...f.....f`.m.fa|
00000060  66 40 66 81 c3 00 02 00  00 e2 ee bb 00 90 8b 77  |f@f............w|
00000070  0a 01 de 4e 89 f0 c1 e8  04 05 00 40 83 e6 0f 8e  |...N.......@....|
00000080  d8 b8 00 0c 05 00 40 8e  c0 bb 08 00 81 c6 00 80  |......@.........|
00000090  8c d8 2d 00 08 8e d8 bf  ff 7f 8c c0 2d 00 08 8e  |..-.........-...|
000000a0  c0 b9 00 80 fd f3 a4 4b  75 e2 8e d9 8e c1 fc fa  |.......Ku.......|
000000b0  49 74 14 e4 64 a8 02 75  f7 b0 d1 e6 64 e4 64 a8  |It..d..u....d.d.|
000000c0  02 75 fa b0 df e6 60 fb  e9 45 13 66 31 c9 66 03  |.u....`..E.f1.f.|
000000d0  44 08 66 83 d1 00 66 51  66 50 66 c1 eb 04 53 6a  |D.f...fQfPf...Sj|
000000e0  00 6a 01 6a 10 89 e6 b4  42 cd 13 72 04 8d 64 10  |.j.j....B..r..d.|
000000f0  c3 88 e0 bf 73 7d e8 49  00 be 67 7d e8 1c 00 be  |....s}.I..g}....|
00000100  81 7d e8 16 00 30 e4 cd  16 c7 06 72 04 34 12 ea  |.}...0.....r.4..|
00000110  00 00 ff ff bb 07 00 b4  0e cd 10 ac 84 c0 75 f4  |..............u.|
00000120  c3 80 fa 80 72 17 bb aa  55 52 b4 41 cd 13 5a 72  |....r...UR.A..Zr|
00000130  0c 81 fb 55 aa 75 06 f6  c1 01 74 01 c3 be 55 7d  |...U.u....t...U}|
00000140  eb ba 50 c0 e8 04 e8 01  00 58 24 0f 3c 0a 1c 69  |..P......X$.<..i|
00000150  2f 0c 20 aa c3 43 48 53  20 6e 6f 74 20 73 75 70  |/. ..CHS not sup|
00000160  70 6f 72 74 65 64 00 52  65 61 64 20 65 72 72 6f  |ported.Read erro|
00000170  72 3a 20 58 58 00 42 6f  6f 74 20 65 72 72 6f 72  |r: XX.Boot error|
00000180  00 0d 0a 00 90 90 90 90  90 90 90 90 90 90 90 90  |................|
00000190  90 90 90 90 90 90 90 90  90 90 90 90 90 90 90 90  |................|
*
000001b0  90 90 90 90 90 90 90 90  90 90 90 90 90 90 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 80 00  |................|
000001f0  01 00 a5 fe ff ff 00 00  00 00 50 c3 00 00 55 aa  |..........P...U.|
00000200

root@test:~ # hd -n 512 -s 524288 /dev/da0s1a | less

00080000  eb 0e 42 54 58 01 02 80  f6 0f b0 06 00 20 00 00  |..BTX........ ..|
00080010  fa 31 c0 8e d0 bc 00 18  8e c0 8e d8 66 6a 02 66  |.1..........fj.f|
00080020  9d bf 00 5e b9 00 19 f3  ab bb 48 95 b9 10 00 bf  |...^......H.....|
00080030  80 00 89 1d 47 47 ab 83  c3 04 e2 f6 bf 00 5e be  |....GG........^.|
00080040  02 96 ac 98 91 e3 1d ac  92 ad 93 ad b6 08 d1 eb  |................|
00080050  73 0b 89 05 88 75 02 88  55 05 83 c0 04 8d 7d 08  |s....u..U.....}.|
00080060  e2 ec eb de c6 45 05 18  c6 45 08 10 c6 45 66 68  |.....E...E...Efh|
00080070  bb 20 28 e8 c5 00 67 0f  01 1d f6 95 00 00 67 0f  |. (...g.......g.|
00080080  01 15 f0 95 00 00 0f 20  c0 40 0f 22 c0 ea 92 90  |....... .@."....|
00080090  08 00 31 c9 b1 10 8e d1  b1 38 0f 00 d9 ba 00 a0  |..1......8......|
000800a0  00 00 36 0f b7 05 13 04  00 00 c1 e0 0a 2d 00 10  |..6..........-..|
000800b0  00 00 29 d0 b1 33 51 50  68 02 02 00 00 6a 2b ff  |..)..3QPh....j+.|
000800c0  35 0c 90 00 00 51 51 51  51 52 b1 07 6a 00 e2 fc  |5....QQQQR..j...|
000800d0  61 07 1f 0f a1 0f a9 cf  fa bc 00 18 00 00 0f 20  |a.............. |
000800e0  c0 25 ff ff ff 7f 0f 22  c0 31 c9 0f 22 d9 2e 0f  |.%.....".1.."...|
000800f0  01 15 f0 95 00 00 66 ea  fc 90 18 00 b1 20 8e d1  |......f...... ..|
00080100  8e d9 8e c1 8e e1 8e e9  48 0f 22 c0 ea 11 91 00  |........H.".....|
00080110  00 31 c0 8e d0 8e d8 bb  08 70 e8 1e 00 67 0f 01  |.1.......p...g..|
00080120  1d fc 95 00 00 fb 67 f6  05 07 90 00 00 01 74 fe  |......g.......t.|
00080130  c7 06 72 04 34 12 ea f0  ff 00 f0 e4 21 50 e4 a1  |..r.4.......!P..|
00080140  50 b0 11 e6 20 e6 a0 88  d8 e6 21 88 f8 e6 a1 b0  |P... .....!.....|
00080150  04 e6 21 b0 02 e6 a1 b0  01 e6 21 e6 a1 58 e6 a1  |..!.......!..X..|
00080160  58 e6 21 c3 6a 00 eb 34  6a 01 eb 30 6a 03 eb 2c  |X.!.j..4j..0j..,|
00080170  6a 04 eb 28 6a 05 eb 24  6a 06 eb 20 6a 07 eb 1c  |j..(j..$j.. j...|
00080180  6a 08 eb 20 6a 0a eb 1c  6a 0b eb 18 6a 0c eb 14  |j.. j...j...j...|
00080190  6a 0d eb 10 6a 0e eb 0c  6a 10 eb 00 ff 34 24 c6  |j...j...j....4$.|
000801a0  44 24 04 00 fc 1e 06 60  0f a8 0f a0 1e 06 66 83  |D$.....`......f.|
000801b0  7c 24 44 08 75 03 16 eb  04 ff 74 24 50 ff 74 24  ||$D.u.....t$P.t$|
000801c0  50 6a 10 1f 1e 07 89 e3  be 21 96 00 00 bf 00 18  |Pj.......!......|
000801d0  00 00 57 e8 56 02 00 00  5e e8 0c 03 00 00 8d 64  |..W.V...^......d|
000801e0  24 18 61 07 1f 80 3c 24  03 74 15 80 3c 24 01 75  |$.a...<$.t..<$.u|
000801f0  0a f7 44 24 10 00 01 00  00 75 05 e9 d8 fe ff ff  |..D$.....u......|
00080200

root@test:~ # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/da0s1a        524288        0   524288     0%


root@test:~ # sysctl -n hw.realmem
536870912

root@test:~ # bc
>>> 536870912/1024/1024
512
>>> quit


root@test:~ # mdconfig -s 512m -u md0
root@test:~ # newfs -U /dev/md0
root@test:~ # mount /dev/md0 /mnt
root@test:~ # dd if=/dev/random of=/mnt/junk.txt
root@test:~ # umount /mnt
root@test:~ # mdconfig -du md0


root@test:~ # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/da0s1a        524288   324216   200072    62%


root@test:~ # hd -n 512 /dev/da0s1a
00000000  fc 31 c9 8e c1 8e d9 8e  d1 bc 00 7c e8 12 01 be  |.1.........|....|
00000010  ee 7d 66 31 c0 66 bb 00  80 00 00 e8 ad 00 b9 01  |.}f1.f..........|
00000020  00 be be 81 b6 01 80 7c  04 a5 75 07 e3 18 f6 04  |.......|..u.....|
00000030  80 75 13 83 c6 10 fe c6  80 fe 05 72 e9 49 e3 e1  |.u.........r.I..|
00000040  be 76 7d e9 b6 00 89 16  00 09 b9 00 02 66 b8 00  |.v}..........f..|
00000050  04 00 00 66 bb 00 90 00  00 66 60 e8 6d 00 66 61  |...f.....f`.m.fa|
00000060  66 40 66 81 c3 00 02 00  00 e2 ee bb 00 90 8b 77  |f@f............w|
00000070  0a 01 de 4e 89 f0 c1 e8  04 05 00 40 83 e6 0f 8e  |...N.......@....|
00000080  d8 b8 00 0c 05 00 40 8e  c0 bb 08 00 81 c6 00 80  |......@.........|
00000090  8c d8 2d 00 08 8e d8 bf  ff 7f 8c c0 2d 00 08 8e  |..-.........-...|
000000a0  c0 b9 00 80 fd f3 a4 4b  75 e2 8e d9 8e c1 fc fa  |.......Ku.......|
000000b0  49 74 14 e4 64 a8 02 75  f7 b0 d1 e6 64 e4 64 a8  |It..d..u....d.d.|
000000c0  02 75 fa b0 df e6 60 fb  e9 45 13 66 31 c9 66 03  |.u....`..E.f1.f.|
000000d0  44 08 66 83 d1 00 66 51  66 50 66 c1 eb 04 53 6a  |D.f...fQfPf...Sj|
000000e0  00 6a 01 6a 10 89 e6 b4  42 cd 13 72 04 8d 64 10  |.j.j....B..r..d.|
000000f0  c3 88 e0 bf 73 7d e8 49  00 be 67 7d e8 1c 00 be  |....s}.I..g}....|
00000100  81 7d e8 16 00 30 e4 cd  16 c7 06 72 04 34 12 ea  |.}...0.....r.4..|
00000110  00 00 ff ff bb 07 00 b4  0e cd 10 ac 84 c0 75 f4  |..............u.|
00000120  c3 80 fa 80 72 17 bb aa  55 52 b4 41 cd 13 5a 72  |....r...UR.A..Zr|
00000130  0c 81 fb 55 aa 75 06 f6  c1 01 74 01 c3 be 55 7d  |...U.u....t...U}|
00000140  eb ba 50 c0 e8 04 e8 01  00 58 24 0f 3c 0a 1c 69  |..P......X$.<..i|
00000150  2f 0c 20 aa c3 43 48 53  20 6e 6f 74 20 73 75 70  |/. ..CHS not sup|
00000160  70 6f 72 74 65 64 00 52  65 61 64 20 65 72 72 6f  |ported.Read erro|
00000170  72 3a 20 58 58 00 42 6f  6f 74 20 65 72 72 6f 72  |r: XX.Boot error|
00000180  00 0d 0a 00 90 90 90 90  90 90 90 90 90 90 90 90  |................|
00000190  90 90 90 90 90 90 90 90  90 90 90 90 90 90 90 90  |................|
*
000001b0  90 90 90 90 90 90 90 90  90 90 90 90 90 90 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 80 00  |................|
000001f0  01 00 a5 fe ff ff 00 00  00 00 50 c3 00 00 55 aa  |..........P...U.|
00000200

root@test:~ # hd -v -n 512 -s 524288 /dev/da0s1a | less

00080000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000800f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080100  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080120  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000801f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00080200

As you see the first sector of 512 bytes doesn't get overwritten but the BTX that is located at 1024sectors * 512bytes = 524288bytes is cleared. That's why your system is unable to boot when you write zfsboot into the freebsd-swap.

Thanks for your help. I did think that swap got cleared/rewritten when new data filled in, but in another thread someone said swapoff -a and swapon -a would do that. So I did the test. Guess I was misled by it. https://forums.freebsd.org/threads/how-to-clear-the-swap.32125/

I don't comprehend all your code but I can see that you dump a large file into swap and it got rewritten. Did you try install FreeBSD in the slice and bootcode in swap like I did? I'll test this when I get the time tomorrow. Will come back with my results.

 

[VladiBG]

The other thread that you are referring is about freeing the swap memory. Here i'm talking of zeroing the swap partition or the swap file. The only way to do this is to turn off the swap using swapoff then to fill the swap partition with zeros using dd(1).

 

[Jaskie]

The other thread that you are referring is about freeing the swap memory. Here i'm talking of zeroing the swap partition or the swap file. The only way to do this is to turn off the swap using swapoff then to fill the swap partition with zeros using dd(1).

Thank for your clarrification. I always thought they were the same thing.

 

[Jaskie]

root@test:~ # hd -n 512 -s 524288 /dev/da0s1a | less
root@test:~ # swapoff -a
root@test:~ # hd -n 512 /dev/da0s1a
root@test:~ # hd -n 512 -s 524288 /dev/da0s1a | less
root@test:~ # swapinfo
root@test:~ # sysctl -n hw.realmem
536870912

root@test:~ # bc
>>> 536870912/1024/1024
512
>>> quit


root@test:~ # mdconfig -s 512m -u md0
root@test:~ # newfs -U /dev/md0
root@test:~ # mount /dev/md0 /mnt
root@test:~ # dd if=/dev/random of=/mnt/junk.txt
root@test:~ # umount /mnt
root@test:~ # mdconfig -du md0

root@test:~ # swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/da0s1a        524288   324216   200072    62%


root@test:~ # hd -n 512 /dev/da0s1a
root@test:~ # hd -v -n 512 -s 524288 /dev/da0s1a | less

As you see the first sector of 512 bytes doesn't get overwritten but the BTX that is located at 1024sectors * 512bytes = 524288bytes is cleared. That's why your system is unable to boot when you write zfsboot into the freebsd-swap.

Tried with this in a VM today and it did result in a unbootable system. And then tried the method you provided above to add another small ZFS partition and install bootcode in it. System did boot again. I think this thread now can be marked as SOLVED now? Thanks for your help as always.