Solved Is it possible to make a gmirror of my GPT file system?

I currently have my file system on ada2 like this:
Code:
# gpart show ada2
=>       34  488397101  ada2  GPT  (233G)
         34          6        - free -  (3.0K)
         40       1024     1  freebsd-boot  (512K)
       1064  480246776     2  freebsd-ufs  (229G)
  480247840    8149294     3  freebsd-swap  (3.9G)
  488397134          1        - free -  (512B)
I also have an identical disk on ada0 which is currently unused. I've been having trouble creating a gmirror using ada0, and according to the handbook

https://www.freebsd.org/doc/handbook/geom-mirror.html

Because GPT partition schemes also store metadata at the end of the disk, mirroring entire GPT disks with gmirror(8) is not recommended.
I think that ada2 is failing because disk accesses have become very slow, so I want to gmirror it and then replace the physical disk with another one when or if it fails. What is the recommended course of action for me?
 
I recommend to make a dump of the /dev/ada2p2 and then to check the surface of the disk for errors and if any errors are found to replace the disk and restore the dump on the new disk.
 
I recommend to make a dump of the /dev/ada2p2
I make weekly backups of the disk using
Code:
/sbin/dump -C16 -b64 -0uanL -h0 -f - /
onto another pair of gmirrored disks. I am currently making one, it says it will be finished on Friday!
Code:
  DUMP: 0.72% done, finished in 68:46 at Fri Oct 18 11:41:21 2019
Hopefully it will get a bit faster.
and then to check the surface of the disk for errors and if any errors are found to replace the disk and restore the dump on the new disk.
What procedure do you recommend for checking the surface of the disk for errors? The command smartctl is not showing any errors in the disk, but the disk has become very slow.
 
Hi,

If you think the disk is failing, first verfiy the diagnosis.

Are you running smartd:
Code:
[sherman.146] $ grep smartd /etc/rc.conf
smartd_enable="YES"
If not, install and start it:
Code:
pkg install smartd
echo 'smartd_enable="YES"'>>/etc/rc.conf 
service smartd start
Then run the report and post the results:
Code:
smartctl -H -a /dev/ada2
If ada2 is failing, you can:
  • create a new mirror, say gm0, on the spare disk, ada0 (with only one side of the mirror populated);
  • populate the mirror from the contents of the file systems on the failing disk, ada2;
  • replace ada2, and configure the new disk as the second side of gm0.
I would also recommend you set aside separate slices for journaling on your new mirror, but let's cross that bridge after we see the smartd report.
 
Hi!
If you think the disk is failing, first verfiy the diagnosis.

Are you running smartd:
Code:
[sherman.146] $ grep smartd /etc/rc.conf
smartd_enable="YES"
Yes, running that.

Then run the report and post the results:
Code:
smartctl -H -a /dev/ada2
I don't think there is anything to see here but here you are:
Code:
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.2-RELEASE-p14 i386] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST250DM000-1BD141
Serial Number:    9VYKP11Z
LU WWN Device Id: 5 000c50 05032912a
Firmware Version: FJK2
User Capacity:    250,059,350,016 bytes [250 GB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Tue Oct 15 15:09:31 2019 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82)    Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:         (  592) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:      (   1) minutes.
Extended self-test routine
recommended polling time:      (  39) minutes.
Conveyance self-test routine
recommended polling time:      (   2) minutes.
SCT capabilities:            (0x303f)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   119   099   006    Pre-fail  Always       -       233646072
  3 Spin_Up_Time            0x0003   100   100   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       70
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   082   060   030    Pre-fail  Always       -       192487861
  9 Power_On_Hours          0x0032   053   053   000    Old_age   Always       -       42012
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       70
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   058   053   045    Old_age   Always       -       42 (Min/Max 24/42)
194 Temperature_Celsius     0x0022   042   047   000    Old_age   Always       -       42 (0 9 0 0 0)
195 Hardware_ECC_Recovered  0x001a   053   034   000    Old_age   Always       -       233646072
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       42013h+05m+02.485s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       21700764
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       1800552204

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The "raw error rate" stuff seems to be a feature of Seagate discs:
If ada2 is failing, you can:
  • create a new mirror, say gm0, on the spare disk, ada0 (with only one side of the mirror populated);
  • populate the mirror from the contents of the file systems on the failing disk, ada2;
  • replace ada2, and configure the new disk as the second side of gm0.
I would also recommend you set aside separate slices for journaling on your new mirror, but let's cross that bridge after we see the smartd report.
I wasn't able to create a new mirror on ada0 using the instructions given in the FreeBSD handbook linked in my initial post, due to a variety of errors. I am not sure if this is related to the issue with the GPT partition system.
 
I have attached the detailed notes I kept the last time I made a gmirror. I had to make some modifications, since my circumstances were a little different (I was installing from release media).

The grand plan is to make ada0 one side of a mirror, copy your data to that, and reboot into that mirror.

The original disk, ada2, should remain in place, and usable until you make the decision that the ada0 mirror is OK. You then replace ada2, and install it as the second side of the mirror.

I had three file systems, plus swap, plus journals, plus a whole second boot environment. But I think you can distill what you need fairly easily.

I hate editing these sorts of plans without testing, so I suggest you wait before using this to see if anyone else can make suggestions for fixes or improvements.

So long as you operate only one unused disk, you should be OK.
Caveat emptor.
 

Attachments

  • gmirror.txt
    6.1 KB · Views: 138
Oh, I would also wait to see what the SMART experts have to say about your report. They should come on line over the next 12 hours or so...
 
Smart data appears to be fine, head-flying-hours is a bit high (close to 5 years) but that's not something to worry about. It does indicate the disk is getting old. You might want to run the short and long tests though, just to make sure.

I think that ada2 is failing because disk accesses have become very slow
How full is the disk? The closer you get to filling up the disk the slower it will get.
 
gmirror requires MBR. You can't use GPT.
This is not correct. You do have to remember that gmirror writes its metadata to the end of the disk, just like GPT does.
Code:
root@X9SRL:~ # gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ada0 (ACTIVE)
                      ada1 (ACTIVE)
root@X9SRL:~ # gpart show
=>      40  31276976  mirror/gm0  GPT  (15G)
        40    204800           1  efi  (100M)
    204840  31072176           2  freebsd-ufs  (15G)
I am using this on my 10G softswitch.
I also use a 3 drive GPT gmirror on another server.
 
Smart data appears to be fine, head-flying-hours is a bit high (close to 5 years) but that's not something to worry about. It does indicate the disk is getting old. You might want to run the short and long tests though, just to make sure.
Thank you for the idea to try the tests.
How full is the disk? The closer you get to filling up the disk the slower it will get.
Not very:
Code:
# df
Filesystem                 1K-blocks       Used     Avail Capacity  Mounted on
/dev/ada2p2                232575284  103384020 110585244    48%    /
But the computer has been running very slowly for a few days, and it is always stuck in disk operations. After I reboot the computer it is OK for a bit, then it starts to get slower and slower. I got some results from diskinfo -tv /dev/ada2 as I found here:


but perhaps the dump I am running caused it to be very slow. I probably should wait until the dump is finished before doing any more operations.
 
gmirror requires MBR. You can't use GPT. But that doesn't matter. You can start afresh with ada0.
So the idea is to use the method described in the above part of the handbook, 18.3.2. Creating a Mirror with Two New Disks, but with a single disk, /dev/ada0,
Code:
# gmirror label -v gm0 /dev/ada0
# gpart create -s MBR mirror/gm0
then copy /dev/ada2p2 as described onto that MBR partition,
Code:
dump -C16 -b64 -0aL -f - / | (cd /mnt && restore -rf -)
and make the other sections as described, then try to boot that? OK, I will try that kind of thing tomorrow when the backup of ada2 is completed.
 
Just an idea, why not move from UFS to ZFS?

Remove ada2 and install FreeBSD with ZFS in ada0, then mount ada2 and copy all user files to ada0, after you are sure all important information is in ada0, configure a ZFS mirror using ada0 and ada2

Check this Thread 49702
 
I did want to mention that the handbook says GPT is not recommended for gmirrors.
Bringing in an existing disk definitely makes it a much harder task.
I made a script that I run from the bsdinstall and the manual disk phase.
 
I got some results from diskinfo -tv /dev/ada2 as I found here:


but perhaps the dump I am running caused it to be very slow. I probably should wait until the dump is finished before doing any more operations.
Here are the results:
Code:
# diskinfo -tv /dev/ada2
/dev/ada2
    512             # sectorsize
    250059350016    # mediasize in bytes (233G)
    488397168       # mediasize in sectors
    4096            # stripesize
    0               # stripeoffset
    484521          # Cylinders according to firmware.
    16              # Heads according to firmware.
    63              # Sectors according to firmware.
    ST250DM000-1BD141    # Disk descr.
    9VYKP11Z        # Disk ident.
    No              # TRIM/UNMAP support
    7200            # Rotation rate in RPM
    Not_Zoned       # Zone Mode

Seek times:
    Full stroke:      250 iter in   9.739405 sec =   38.958 msec
    Half stroke:      250 iter in  11.517978 sec =   46.072 msec
    Quarter stroke:      500 iter in  19.603323 sec =   39.207 msec
    Short forward:      400 iter in  20.390776 sec =   50.977 msec
    Short backward:      400 iter in  18.007958 sec =   45.020 msec
    Seq outer:     2048 iter in  54.766048 sec =   26.741 msec
    Seq inner:     2048 iter in  92.657058 sec =   45.243 msec

Transfer rates:
    outside:       102400 kbytes in  36.082337 sec =     2838 kbytes/sec
    middle:        102400 kbytes in  45.280484 sec =     2261 kbytes/sec
    inside:        102400 kbytes in  47.667309 sec =     2148 kbytes/sec
For comparison, here are the results of /dev/ada0 (an identical disk):
Code:
# diskinfo -tv /dev/ada0
/dev/ada0
    512             # sectorsize
    250059350016    # mediasize in bytes (233G)
    488397168       # mediasize in sectors
    4096            # stripesize
    0               # stripeoffset
    484521          # Cylinders according to firmware.
    16              # Heads according to firmware.
    63              # Sectors according to firmware.
    ST250DM000-1BD141    # Disk descr.
    9VYKNS4C        # Disk ident.
    No              # TRIM/UNMAP support
    7200            # Rotation rate in RPM
    Not_Zoned       # Zone Mode

Seek times:
    Full stroke:      250 iter in   7.029522 sec =   28.118 msec
    Half stroke:      250 iter in   5.097744 sec =   20.391 msec
    Quarter stroke:      500 iter in   7.990762 sec =   15.982 msec
    Short forward:      400 iter in   3.082881 sec =    7.707 msec
    Short backward:      400 iter in   3.143737 sec =    7.859 msec
    Seq outer:     2048 iter in   0.121591 sec =    0.059 msec
    Seq inner:     2048 iter in   0.126456 sec =    0.062 msec

Transfer rates:
    outside:       102400 kbytes in   0.741227 sec =   138149 kbytes/sec
    middle:        102400 kbytes in   0.839750 sec =   121941 kbytes/sec
    inside:        102400 kbytes in   1.353097 sec =    75678 kbytes/sec
I think ada2 is running very slowly.

(Edit: added ada0 information for comparison.)
 
Just an idea, why not move from UFS to ZFS?
Thanks for the idea. One reason is that it is still a 32-bit system at the moment. Another is that I think it's safer to take things one step at a time, so first of all get this back into working order before making more drastic changes.
 
Here is what I have done so far, and the outputs, following the handbook.

https://www.freebsd.org/doc/handbook/geom-mirror.html

Code:
# gmirror label -v gm0 /dev/ada0
Metadata value stored on /dev/ada0.
Done.
# ls /dev/mirror
gm0    gm1    gm1s1    gm1s1a
# gpart create -s MBR mirror/gm0
mirror/gm0 created
# gpart add -t freebsd -a 4k mirror/gm0
mirror/gm0s1 added
# gpart show mirror/gm0
=>       63  488397104  mirror/gm0  MBR  (233G)
         63          1              - free -  (512B)
         64  488397096           1  freebsd  (233G)
  488397160          7              - free -  (3.5K)

# gpart create -s BSD mirror/gm0s1
mirror/gm0s1 created
# gpart add -t freebsd-ufs -a 4k -s 229g mirror/gm0s1
mirror/gm0s1a added
# gpart add -t freebsd-swap -a 4k -s 4g mirror/gm0s1
gpart: autofill: No space left on device
# gpart add -t freebsd-swap -a 4k -s 3.9g mirror/gm0s1
gpart: Invalid size param: Invalid argument
# gpart show mirror/gm0
=>       63  488397104  mirror/gm0  MBR  (233G)
         63          1              - free -  (512B)
         64  488397096           1  freebsd  (233G)
  488397160          7              - free -  (3.5K)

# gpart delete mirror/gm0s1a
gpart: Option 'i' not specified.
usage: gpart add -t type [-a alignment] [-b start] [-s size] [-i index] [-l label] [-f flags] geom
       gpart backup geom
<trim>
# gpart destroy mirror/gm0s1
gpart: Device busy
# gpart destroy -F mirror/gm0s1
mirror/gm0s1 destroyed
# gpart create -s BSD mirror/gm0s1
mirror/gm0s1 created
# gpart add -t freebsd-ufs -a 4k -s 220g mirror/gm0s1
mirror/gm0s1a added
# gpart add -t freebsd-swap -a 4k -s 4g mirror/gm0s1
mirror/gm0s1b added
# gpart show mirror/gm0s1
=>        0  488397096  mirror/gm0s1  BSD  (233G)
          0  461373440             1  freebsd-ufs  (220G)
  461373440    8388608             2  freebsd-swap  (4.0G)
  469762048   18635048                - free -  (8.9G)

# gpart bootcode -b /boot/mbr mirror/gm0
bootcode written to mirror/gm0
# gpart set -a active -i 1 mirror/gm0
active set on mirror/gm0s1
# gpart bootcode -b /boot/boot mirror/gm0s1
bootcode written to mirror/gm0s1
# newfs -U /dev/mirror/gm0s1a
/dev/mirror/gm0s1a: 225280.0MB (461373440 sectors) block size 32768, fragment size 4096
    using 360 cylinder groups of 626.22MB, 20039 blks, 80256 inodes.
    with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 1282688, 2565184, 3847680, 5130176, 6412672, 7695168, 8977664, 10260160,
<trim>
 460416256
# mount /dev/mirror/gm0s1a /monty
mount: /monty: No such file or directory
# mkdir /monty
# mount /dev/mirror/gm0s1a /monty
# dump -C16 -b64 -0aL -f - / | (cd /monty && restore -rf -)
I'm just waiting now.
 
I also wanted to call your attention to some other geom goodies.
Notably graid3(8) and UFS's software RAID5, not in base but a port sysutils/graid5.
ZFS gets a lion share of the attention but Netflix uses UFS so it is not as bad as some make it out to be..
Plus it is easier to use in my opinion. Great for your 32-bit platform with only 4GB ram addressable.
 
OK the dump finished at last, and I edited /monty/etc/fstab as follows
Code:
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/gm0s1a             /               ufs     rw              1       1
/dev/gm0s1b             /               swap    sw              0       0
<trim>
and tried booting without success (Error 19), but re-reading the instructions I got it right the second time:
Code:
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/mirror/gm0s1a              /               ufs     rw              1       1
/dev/mirror/gm0s1b              none            swap    sw              0       0
<trim>
and I'm now using the rebooted system successfully.
Code:
$ mount
/dev/mirror/gm0s1a on / (ufs, NFS exported, local, soft-updates)
devfs on /dev (devfs, local, multilabel)
fdescfs on /dev/fd (fdescfs)
/dev/mirror/gm1s1a on /mnt (ufs, local, soft-updates)
So I wasn't able to use the GPT partition, but I did solve my problem. Thank you everyone. :)
 
Back
Top