Solved UFS RAID1 - Help

C

cutter

I am trying to create a RAID1 with two new disks on IBM x3250 M5 (10.0-RELEASE does not have a driver for the existing RAID controller) with no success. There are errors which I don't know how to deal with. And this is critical. So please help. The steps I have done so far as described in the manual:

I disabled the soft journaling with tunefs in single user mode.
The two disks are ada0 and ada1
The operating system installed cleanly with BSD labels on ada0.

Code: 
gmirror load
diskinfo -v ada0 | head -n3
ada0
  512  # sectorsize
  1000204886016  # mediasize in bytes (932G)

Code: 
 gpart show ada0
=>  0  1953525168  ada0  BSD  (932G)
  0  8388608  1  freebsd-ufs  (4.0G)
  8388608  8388608  2  freebsd-swap  (4.0G)
  16777216  2097152  4  freebsd-ufs  (1.0G)
  18874368  16777216  5  freebsd-ufs  (8.0G)
  35651584  1917873583  6  freebsd-ufs  (915G)
  1953525167  1  - free -  (512B)

Code: 
geom zero load
gnop create -s 1000204886016 gzero
# sysctl kern.geom.debugflags=16
# gmirror label -v gm0 gzero.nop ada1
# gmirror forget gm0

gpart backup ada0 > table.ada0

So far so good.

Code: 
#gpart restore mirror/gm0 < table.ada0
gpart: geom 'mirror/gm0': File exists
And this is the point where I am stuck. I deleted the content of mirror/gm0 and tried again and received the same error. How can I get over this error?

Thank you for your help in advance.
 
Partitioning should be GPT or MBR. The BSD labels are for inside a partition aka slice. You seem to have inadvertently created a "dangerously dedicated" layout.
 
SirDice said:
Partitioning should be GPT or MBR. The BSD labels are for inside a partition aka slice. You seem to have inadvertently created a "dangerously dedicated" layout.
Click to expand...

I installed a clean new system with MBR and BSD slice. The same problem persists. Moreover, there is no gm0s1 file in the /dev/mirror/ directory. The content of this directory is:
Code: 
crw-r-----  1 root  operator  0xa5 Nov 18 15:54 gm0
crw-r-----  1 root  operator  0xa6 Nov 18 15:54 gm0a
crw-r-----  1 root  operator  0xa7 Nov 18 15:54 gm0b
crw-r-----  1 root  operator  0xa8 Nov 18 15:54 gm0d
crw-r-----  1 root  operator  0xa9 Nov 18 15:54 gm0e
crw-r-----  1 root  operator  0xaa Nov 18 15:54 gm0f
Any other ideas?
 
wblock@ said:
Again, you are using only bsdlabel(8) partitioning. See this thread for creating MBR partitioning: Thread creating-a-traditional-mbr-layout-with-bsdinstall.45072.
Click to expand...

I did exactly as it is advised there, but of no avail. Here is the output of gpart:

Code: 
# gpart show ada0
=>  63  1953525105  ada0  MBR  (932G)
       63  1953525042  1  freebsd  [active]  (932G)
       1953525105  63  - free -  (32K)

# cat table.ada0s1
BSD 8
1  freebsd-ufs  0  8388608
2 freebsd-swap  8388608  8388608
4  freebsd-ufs  16777216  2097152
5  freebsd-ufs  18874368  16777216
6  freebsd-ufs  35651584 1917873457

So this does not seem to be the problem.
 
You are making progress. If you still see an error when trying to restore a partition table to the new mirror, it is because there is already a partition table there. Use the -F option to force overwriting it, but make very sure it is on the correct drive.

You made a full backup before starting, right?
 
Thanks!
The mirror disk has some partitions from the previous attempts when skipping the error message. Tomorrow I will go on with your suggestion.
 
wblock@ said:
You are making progress. If you still see an error when trying to restore a partition table to the new mirror, it is because there is already a partition table there. Use the -F option to force overwriting it, but make very sure it is on the correct drive.

You made a full backup before starting, right?
Click to expand...

Now I got over the creating of the array and even succeeded to boot from the second drive. I have one more (final) hurdle to overcome. When I tried to synchronize both disks and I got the following error:

Code: 
# df -h

Filesystem                 Size   Used    Avail Capacity  Mounted on
/dev/mirror/gm0s1a  3.9G  475M  3.1G      13%     /
devfs                         1.0K   1.0K    0B         100%  /dev
/dev/mirror/gm0s1d  992M  8.1M  904M     1%      /tmp
/dev/mirror/gm0s1e  7.7G   33M    7.1G      0%      /var
/dev/mirror/gm0s1f  886G   2.0G   813G     0%      /usr

#gmirror insert gm0 ada0
gmirror: Not all disks connected.

# gmirror status
  Name  Status  Components
mirror/gm0  DEGRADED  ada0 (ACTIVE)

# gpart show ada0
gpart: No such geom: ada0

Any suggestions from the esteemed forum what might possibly have gone wrong and how to overcome it?

Thanks.
 
It looks like ada0 was already in the mirror. Without seeing gmirror status from before that step, it's hard to tell.

Drive numbers here are dynamic. If you physically swapped the disks, ada0 and ada1 also swapped.
 
wblock@ said:
It looks like ada0 was already in the mirror. Without seeing gmirror status from before that step, it's hard to tell.

Drive numbers here are dynamic. If you physically swapped the disks, ada0 and ada1 also swapped.
Click to expand...

I haven't swapped the disks. Any suggestions how to troubleshoot it? The only thing which comes to my mind is to starts again from a scratch.
 
wblock@ said:
Starting again is the only thing I can suggest. As noted in the Handbook, once a drive is added to a mirror, any information on it is overwritten by the mirror.

Before starting, reread the section on metadata: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html#geom-mirror-metadata.
Click to expand...

I succeeded in destroying the metadata only on ada1. I repeated the whole procedure from a scratch, leaving even more space at the end of the disk. And again I had the same error. This time I recorded the status before issuing the directive. Here are the outputs:
Code: 
# gmirror status
  Name  Status  Components
mirror/gm0  DEGRADED  ada0 (ACTIVE)
# gmirror insert gm0 ada0
gmirror: Not all disks connected.
# gmirror rebuild gm0 ada0
gmirror: Provider ada0 is the last active provider in gm0.

What do you think? Possible solutions?
 
I still don't see why ada0 is already in the mirror. The Handbook procedure for creating a mirror with only one new disk creates the mirror on the new disk (ada1 in this case), then copies data from the old disk (ada0 here) onto that new mirror. After the new mirror is set up on that one new disk and checked, then the old disk (ada0) is added to the mirror.

The output shows that ada0 is already in a mirror, and ada1 is not shown at all. I expected gmirror status to show only ada1.
 
wblock@ said:
I still don't see why ada0 is already in the mirror. The Handbook procedure for creating a mirror with only one new disk creates the mirror on the new disk (ada1 in this case), then copies data from the old disk (ada0 here) onto that new mirror. After the new mirror is set up on that one new disk and checked, then the old disk (ada0) is added to the mirror.
The output shows that ada0 is already in a mirror, and ada1 is not shown at all. I expected gmirror status to show only ada1.
Click to expand...

I installed the OS anew, disabled soft updates journaling and again repeated the whole procedure word by word from the manual. And again I received

Code: 
# gmirror status
Name Status Components
mirror/gm0 DEGRADED ada0 (ACTIVE)

ada1 is not existing. In few words - I am totally lost.
 
You might still have metadata on ada0 also.

This is the link to the current version of the instructions:
https://www.freebsd.org/doc/en_US.I...k/geom-mirror.html#geom-mirror-existing-drive

Both drives are connected, and /dev/ada0 and /dev/ada1 exist.

When the operating system is installed, you are not using ZFS or encryption (probably not, but just to be sure).

Before starting, old metadata on both drives must be cleared. That is both the old partition tables ( gpart destroy -F) and the mirror metadata ( gmirror clear).

After creating the mirror with gmirror label -v gm0 gzero.nop ada1 and gmirror forget gm0, stop. What does gmirror status say at that point?
 
Thanks for your help and advice.

Here is the output from the clearing of the metadata:
Code: 
# gpart destroy -F ada1
ada1 destroyed

# gpart destroy -F ada0
gpart: Device busy

# gmirror clear ada0
Can't clear metadata on ada0: Operation not permitted.
gmirror: Not fully done.
And after creating a mirror with the new disk:
Code: 
# diskinfo -v ada0 | head -n3
ada0
  512  # sectorsize
  1000204886016  # mediasize in bytes (932G)
# geom zero load
# gnop create -s 1000204886016 gzero
# gmirror label -v gm0 gzero.nop ada1
Metadata value stored on gzero.nop.
Metadata value stored on ada1.
Done.
# gmirror forget gm0
# gmirror status
  Name  Status  Components
mirror/gm0  COMPLETE  ada1 (ACTIVE)
Actually the metadata on ada0 is unreachable. I tried to clear it in singe user mode:
Code: 
#gpart destroy -F ada0
gpart: Device busy
# gmirror clear ada0
Only the last directive worked without a glitch. gmirror status gave a segmentation fault. Now I'll try again and let's see what will be the result.
 
The ada1 mirror looks perfect there.

A device can be removed from a running mirror and the metadata cleared with gmirror remove. I suspect that ada0 had that metadata and the mirror name was the same, hiding the new mirror you were trying to create.

As a last resort only, that "debugflags" sysctl(8) in your first post should allow clearing of metadata that is in use.
 
Solved.
After a week of effort, we did it. The critical point was clearing the metadata from ada0.
Thanks.
 
You must log in or register to reply here.
Back
Top