Question about /efiboot & /efigpt

freezr

freezr

I've just re-installed FreeBSD with some hiccups, on that machine I had three disks, two were used for ZFS vdev mir, and the third one was used for Linux.
However the installation created /efiboot & /efigpt across all the three disks. Is that the normal behavior? ?‍♂️

Thanks ?
 
I have a somewhat outdated 13.1-RELEASE but it should not be so much far behind the current 13.2 one.
My system does not have anything named efiboot or efigpt on it. Can it be that some other OS, e.g. Linux, has created that?
What is your partition and ZFS dataset configuration for those disks? Root, boot? When you write the files were created "across all the three disks", which partitions/datasets were the files written to?
 
If you're booting from a mirror set it would be wise to add the boot code to both disks. You still want the system to boot if one of the disks dies.
 
D-FENS said:
I have a somewhat outdated 13.1-RELEASE but it should not be so much far behind the current 13.2 one.
My system does not have anything named efiboot or efigpt on it. Can it be that some other OS, e.g. Linux, has created that?
What is your partition and ZFS dataset configuration for those disks? Root, boot? When you write the files were created "across all the three disks", which partitions/datasets were the files written to?
Click to expand...

I followed the default installer setup, no fancy settings, just mirrored the disk.
 
SirDice said:
If you're booting from a mirror set it would be wise to add the boot code to both disks. You still want the system to boot if one of the disks dies.
Click to expand...

This make senses but it wrote also on the third disk and during the installation process I saw the installer operating on the third disk even though it wasn't selected. This is not a big deal it took only 500MB on each disk, I found just it unusual.
 
freezr said:
This make senses but it wrote also on the third disk and during the installation process I saw the installer operating on the third disk even though it wasn't selected. This is not a big deal it took only 500MB on each disk, I found just it unusual.
Click to expand...
If the installer can create/manage RAID (including mirror) configuration, it would be the safest prediction that all are used for RAID.
For this case, as which physical drive dies first is not known, keeping all physical drive bootable would be the best prediction. Especially all physical disks are of the same capacity and geometry.
 
freezr said:
I followed the default installer setup, no fancy settings, just mirrored the disk.
Click to expand...
But what is the output of zpool status, gpart show etc. though? It is yet unclear if the creator of that particular files is FreeBSD or Linux and where exactly are they located.
In which dataset / partition are the files exactly?
 
Yes, a little more information please: Glad you noticed and said something. Here is a list of 3 commands that are useful to share all the informational details on disk drives to understand your setup.
camcontrol devlist
geom disk list
gpart show -lp
gpart show -rp
gpart show -r
 
wb7odyfred said:
Yes, a little more information please: Glad you noticed and said something...
Click to expand...

Hi, here I am!
Just a little note: FreeBSD is installed on the Samsung disks, Devuan is on the Kingston one.

geom disk list
Code: 
Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r2w2e5
   descr: Samsung SSD 860 EVO M.2 250GB
   lunid: 5002538e31c02be1
   ident: S413NS0RC00973X
   rotationrate: 0
   fwsectors: 63
   fwheads: 16

Geom name: ada1
Providers:
1. Name: ada1
   Mediasize: 240057409536 (224G)
   Sectorsize: 512
   Mode: r1w1e1
   descr: KINGSTON SA400S37240G
   lunid: 50026b7682359175
   ident: 50026B7682359175
   rotationrate: 0
   fwsectors: 63
   fwheads: 16

Geom name: ada2
Providers:
1. Name: ada2
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r0w0e0
   descr: Samsung SSD 860 EVO 250GB
   lunid: 5002538e40f0f259
   ident: S3YHNX0M415404N
   rotationrate: 0
   fwsectors: 63
   fwheads: 16

gpart show -lp
Code: 
>       40  488397088    ada0  GPT  (233G)
         40     532480  ada0p1  efiboot0  (260M)
     532520       1024  ada0p2  gptboot0  (512K)
     533544        984          - free -  (492K)
     534528  487862272  ada0p3  zfs0  (233G)
  488396800        328          - free -  (164K)

=>       40  468862048    ada1  GPT  (224G)
         40     532480  ada1p1  efiboot0  (260M)
     532520       1024  ada1p2  gptboot0  (512K)
     533544        984          - free -  (492K)
     534528   15624192  ada1p3  (null)  (7.5G)
   16158720  452702208  ada1p4  Devuan  (216G)
  468860928       1160          - free -  (580K)

=>       40  488397088    ada2  GPT  (233G)
         40     532480  ada2p1  efiboot1  (260M)
     532520       1024  ada2p2  gptboot1  (512K)
     533544  487863584          - free -  (233G)

gpart show -rp
Code: 
=>       40  488397088    ada0  GPT  (233G)
         40     532480  ada0p1  c12a7328-f81f-11d2-ba4b-00a0c93ec93b  (260M)
     532520       1024  ada0p2  83bd6b9d-7f41-11dc-be0b-001560b84f0f  (512K)
     533544        984          - free -  (492K)
     534528  487862272  ada0p3  516e7cba-6ecf-11d6-8ff8-00022d09712b  (233G)
  488396800        328          - free -  (164K)

=>       40  468862048    ada1  GPT  (224G)
         40     532480  ada1p1  c12a7328-f81f-11d2-ba4b-00a0c93ec93b  (260M)
     532520       1024  ada1p2  83bd6b9d-7f41-11dc-be0b-001560b84f0f  (512K)
     533544        984          - free -  (492K)
     534528   15624192  ada1p3  0657fd6d-a4ab-43c4-84e5-0933c84b4f4f  (7.5G)
   16158720  452702208  ada1p4  0fc63daf-8483-4772-8e79-3d69d8477de4  (216G)
  468860928       1160          - free -  (580K)

=>       40  488397088    ada2  GPT  (233G)
         40     532480  ada2p1  c12a7328-f81f-11d2-ba4b-00a0c93ec93b  (260M)
     532520       1024  ada2p2  83bd6b9d-7f41-11dc-be0b-001560b84f0f  (512K)
     533544  487863584          - free -  (233G)

gpart show -r
Code: 
=>       40  488397088  ada0  GPT  (233G)
         40     532480     1  c12a7328-f81f-11d2-ba4b-00a0c93ec93b  (260M)
     532520       1024     2  83bd6b9d-7f41-11dc-be0b-001560b84f0f  (512K)
     533544        984        - free -  (492K)
     534528  487862272     3  516e7cba-6ecf-11d6-8ff8-00022d09712b  (233G)
  488396800        328        - free -  (164K)

=>       40  468862048  ada1  GPT  (224G)
         40     532480     1  c12a7328-f81f-11d2-ba4b-00a0c93ec93b  (260M)
     532520       1024     2  83bd6b9d-7f41-11dc-be0b-001560b84f0f  (512K)
     533544        984        - free -  (492K)
     534528   15624192     3  0657fd6d-a4ab-43c4-84e5-0933c84b4f4f  (7.5G)
   16158720  452702208     4  0fc63daf-8483-4772-8e79-3d69d8477de4  (216G)
  468860928       1160        - free -  (580K)

=>       40  488397088  ada2  GPT  (233G)
         40     532480     1  c12a7328-f81f-11d2-ba4b-00a0c93ec93b  (260M)
     532520       1024     2  83bd6b9d-7f41-11dc-be0b-001560b84f0f  (512K)
     533544  487863584        - free -  (233G)

Quick note: this partition is the Linux swap that I am sharing with with Devuan.
Code: 
     534528   15624192  ada1p3  (null)  (7.5G)

Thanks! ?
 
If I assume correctly, /efiboot and /efigpt are located in ada1p4 and ada0p3, right? Also assuming that the files would be in whichever dataset you mount as root on your ZFS pool on ada0, we still don't have the name of the pool.
This would be weird, I would not expect anything to appear that "across all 3 disks".
Also, let me point out that your third disk ada2 is empty (you have 233 Gigs free without a partition on it). You have not created a partition to mirror or stripe with ada0. Is this intentional?

Another assumption would be that the files you mentioned are actually written to the EFI partitions: ada0p1, ada1p1, ada2p1.
In this case they would be some kind of EFI boot loader code, but I could not find any information on such files.

Lastly, check the date and timestamp of those files - when have they been created. Do you remember what exactly did you do at that particular time? If the time matches whenever you installed Devuan, this would probably meen that the files were created by the installation.

You could use an online malware scanner to upload them and see if anything funky is going on.
 
freezr said:
Quick note: this partition is the Linux swap that I am sharing with with Devuan.
Code: 
     534528   15624192  ada1p3  (null)  (7.5G)
Click to expand...
I think I tried sharing swap in the past between FreeBSD and Debian. It didn't go well. There were signatures on the swap space that caused problems at boot. But that was ages ago.

A more significant issue is that you have mirror'd your system with ZFS, but not mirror'd the swap space. So if a disk fails, and swap is in use, my guess is that your kernel panics. Not the best outcome...

Last time I built a physical FreeBSD ZFS system, I mirror'd swap manually using gmirror:
Code: 
[sherman.152] $ grep swap /etc/fstab
#/dev/mirror/swap      none        swap    sw        0    0
# With ".eli" appeded to the swap device, swapon(8) will set up GELI encrypt.
/dev/mirror/swap.eli      none        swap    sw        0    0
[sherman.153] $ swapinfo   
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/swap.eli  16777212    34924 16742288     0%
[sherman.154] $ gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  ada0p2 (ACTIVE)
                       ada1p2 (ACTIVE)
If you ever decide to re-do the layout, I'd add the swap gmirror, as you have the space.

My swap is on SSD, and swap is not TRIM'd. It's not used at all often, so I am not concerned about an undesirable duty cycle. Everything else is TRIM'd. All TRIM does is tell the SSD controller about the contents of the kernel's free list. TRIM'ing the relatively small amount of swap is not essential (assuming the majority of your SSD is TRIM'd, and your SSDs have the normal amount of factory over-provisioning).
 
gpw928

Thanks for your advice, this is the first time I read about gmirror... but wouldn't be the same by just doing a 1GB swap file, as a fallback? ?
 
D-FENS said:
If I assume correctly, /efiboot and /efigpt are located in ada1p4 and ada0p3, right? Also assuming that the files would be in whichever dataset you mount as root on your ZFS pool on ada0, we still don't have the name of the pool.
This would be weird, I would not expect anything to appear that "across all 3 disks".
Also, let me point out that your third disk ada2 is empty (you have 233 Gigs free without a partition on it). You have not created a partition to mirror or stripe with ada0. Is this intentional?

Another assumption would be that the files you mentioned are actually written to the EFI partitions: ada0p1, ada1p1, ada2p1.
In this case they would be some kind of EFI boot loader code, but I could not find any information on such files.

Lastly, check the date and timestamp of those files - when have they been created. Do you remember what exactly did you do at that particular time? If the time matches whenever you installed Devuan, this would probably meen that the files were created by the installation.

You could use an online malware scanner to upload them and see if anything funky is going on.
Click to expand...

Now that you wrote this I believe that is very weird, that disk that is "empty" should be part of the mirror.
It looks like the installation failed and I didn't even notice it...

Code: 
zpool status                                                                                                                  2.497s 23:08
  pool: zroot
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
    the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-2Q
config:

    NAME            STATE     READ WRITE CKSUM
    zroot           DEGRADED     0     0     0
      mirror-0      DEGRADED     0     0     0
        ada0p3.eli  ONLINE       0     0     0
        ada2p3.eli  UNAVAIL      0     0     0  cannot open


?
 
Not sure what happened... But the partition is missing:

Code: 
ls /dev/ | grep ada                             01:44
ada0
ada0p1
ada0p2
ada0p3
ada0p3.eli
ada1
ada1p1
ada1p2
ada1p3
ada1p4
ada2
ada2p1
ada2p2

So the installer messed up everything, created triple partitions, did not formatted and create the partition for the mirrored disk...

If I encrypt the disk, it will use the same geli password?

Don't tell me I must reinstall everything? Will never have the time for another fresh install...?
 
freezr said:
gpw928

Thanks for your advice, this is the first time I read about gmirror... but wouldn't be the same by just doing a 1GB swap file, as a fallback? ?
Click to expand...
Swaps are meant for handling small peaks in memory usage that exceed the available RAM. There may be some borderline cases that would need swap mirroring but in most cases it is a waste in my opinion. I am no expert on swap by any means, just common sense.
In this regard, why would you want to waste half your RAM in mirroring the information in the memory? Maybe if you are concerned about memory reliability in some kind of military scenario... perhaps.
 
freezr said:
Not sure what happened... But the partition is missing:

Code: 
ls /dev/ | grep ada                             01:44
ada0
ada0p1
ada0p2
ada0p3
ada0p3.eli
ada1
ada1p1
ada1p2
ada1p3
ada1p4
ada2
ada2p1
ada2p2

So the installer messed up everything, created triple partitions, did not formatted and create the partition for the mirrored disk...

If I encrypt the disk, it will use the same geli password?

Don't tell me I must reinstall everything? Will never have the time for another fresh install...?
Click to expand...
No, don't worry. This is fixable easily. In the meantime the only issue is that your ZFS pool does not have any redundancy, so as long as you don't have any disk errors you are fine.

Simply use gpart(8) to create a new identical parition on ada2. Make sure the size in bytes is the same, this is a must.
Then use geli(8) to encrypt it and make sure it has the same flags as the ada0p3 (use geli to analyze the configuration of the partition, e.g. geli status etc.). Pay attention to any usage of the "-b" and "-g" flags regarding booting, I don't remember by heart.

Here is some commands I use as examples:
Bash: 
# Create a partition for ZFS root pool on da4
gpart add -t freebsd-zfs -l zfs4 -b 2048 -s 60059648 da4

# Encrypt a partition, which will DELETE any information on it.
# You may or may not want to use a keyfile, this is optional
# My ZFS pool is NOT mounted at boot. In your case maybe add a "-g" flag, see in the documentation!
geli init -e AES-XTS -l 256 -s 4096 -b -K /root/keys/storage.key gpt/storage0

# Open an encrypted partition for usage
geli attach -k /root/keys/storage.key gpt/storage0

# Close
geli detach gptid/<guid>.eli

Then you can tell your ZFS pool that its VDEV is now online: zpool online ada2p3.eli and, fingers crossed, it will resilver it.
Once resilvered, hit a zpool scrub zroot to be on the safe side.

If for some reason the VDEV changes and it's not ada2p3 anymore,don't panic. You can just attach a new VDEV to the mirror. DO NOT USE zpool add!!! This would make a stripe and you don't want that. That would be irreversible and you would have to reinstall everything. Just use zpool attach, which would make your mirror into a 3-VDEV mirror. You can simply detach the VDEV that is not ONLINE and all would be fine.
 
Seeing that one of your paritions disappeared after installing, I would bet my money on Devuan or some other partitioning tool deleting the partition without FreeBSD knowing about it.
It is quite improbable that FreeBSD installed incorrectly. I cannot imagine how would this happen.
This would probably be also the explanation of the weird files you are asking about, it may be some kind of buggy installation tool independent from FreeBSD, I assume.
 
D-FENS said:
Swaps are meant for handling small peaks in memory usage that exceed the available RAM. There may be some borderline cases that would need swap mirroring but in most cases it is a waste in my opinion. I am no expert on swap by any means, just common sense.
Click to expand...
You may choose to configure swap, or not, as you wish. The conventional advice is to configure it, because disk space is cheap, and swap covers the contingency if you ever need more virtual memory than you have RAM. i.e. swap will keep your system running normally if this happens.

If you choose to configure swap, and mirror your file systems, you should mirror your swap. To do otherwise will reduce the probability that your system will survive a disk failure.
D-FENS said:
In this regard, why would you want to waste half your RAM in mirroring the information in the memory? Maybe if you are concerned about memory reliability in some kind of military scenario... perhaps.
Click to expand...
I can't decipher what you mean by that.
 
D-FENS said:
No, don't worry. This is fixable easily. In the meantime the only issue is that your ZFS pool does not have any redundancy, so as long as you don't have any disk errors you are fine[...]
Click to expand...

I had better read this before to start, in fact I can't mount the second disk at boot... ?

Well, I guess I must remove the disk and formatting it again, what I do not want is having a key. Is there a way to auto-decrypt two disks giving the passphrase only once (for both)?

Thanks! ?
 
D-FENS said:
Seeing that one of your paritions disappeared after installing, I would bet my money on Devuan or some other partitioning tool deleting the partition without FreeBSD knowing about it.
It is quite improbable that FreeBSD installed incorrectly. I cannot imagine how would this happen.
This would probably be also the explanation of the weird files you are asking about, it may be some kind of buggy installation tool independent from FreeBSD, I assume.
Click to expand...

Did I erase the partition without noticing? I don't exclude that but the disks were labeled, Samsung and Kingston but I am a master of pebcak and anything can happen... ?
 
Looks like I fixed my issue, the pool is properly mount, and I don't need to to insert the passphrase twice!

Code: 
 zpool status                                                                                                                         01:22
  pool: zroot
 state: ONLINE
  scan: resilvered 107G in 00:05:55 with 0 errors on Sat Oct 14 01:14:43 2023
config:

    NAME            STATE     READ WRITE CKSUM
    zroot           ONLINE       0     0     0
      mirror-0      ONLINE       0     0     0
        ada0p3.eli  ONLINE       0     0     0
        ada2p3.eli  ONLINE       0     0     0

errors: No known data errors
 
gpw928 said:
You may choose to configure swap, or not, as you wish. The conventional advice is to configure it, because disk space is cheap, and swap covers the contingency if you ever need more virtual memory than you have RAM. i.e. swap will keep your system running normally if this happens.

If you choose to configure swap, and mirror your file systems, you should mirror your swap. To do otherwise will reduce the probability that your system will survive a disk failure.

I can't decipher what you mean by that.
Click to expand...
Oh, I get it. If you want to keep the system running after a disk failure, it makes sense. For my purposes it has always been fine to crash in such case. Cheers.
Keep in mind that you sacrifice half your swap performance for that. If you put half swap on each disk, the speed is double that of the mirrored one.
 
You must log in or register to reply here.
Back
Top