new geom_mirror array.

R

robertclemens

I have just added 2 hard drives to my 7.2-RELEASE-p4 server.
These drives are identical WD 500GB 7200 SATA drives.
The server is a Tyan GX28 (B2881G28S4H if you want the specific model for any reason).

I;m having trouble getting the system to format these correctly. I sadly have relied on sysinstall too much for my drive partitioning/labeling and I can't quite seem to get this gmirror 100% working.

Now on the the specifics of the problem!
*** OH! And please note that mirror/gm0 works perfectly and I'm only needing any changes/help to mirror/500GB. ***

Code: 
[root@abyss ~]# gmirror status
        Name    Status  Components
  mirror/gm0  COMPLETE  ad4
                        ad6
mirror/500GB  COMPLETE  ad8
                        ad10
[root@abyss ~]#

Code: 
[root@abyss ~]# gmirror list
Geom name: gm0
State: COMPLETE
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 2
ID: 3317308697
Providers:
1. Name: mirror/gm0
   Mediasize: 320072932864 (298G)
   Sectorsize: 512
   Mode: r6w6e7
Consumers:
1. Name: ad4
   Mediasize: 320072933376 (298G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   GenID: 0
   SyncID: 2
   ID: 3601985553
2. Name: ad6
   Mediasize: 320072933376 (298G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   GenID: 0
   SyncID: 2
   ID: 4042299093

Geom name: 500GB
State: COMPLETE
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 3601984412
Providers:
1. Name: mirror/500GB
   Mediasize: 500107861504 (466G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: ad8
   Mediasize: 500107862016 (466G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 1809492736
2. Name: ad10
   Mediasize: 500107862016 (466G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 3231205766

[root@abyss ~]#

Code: 
[root@abyss /dev/mirror]# ls -lah
total 1
dr-xr-xr-x  2 root  wheel          512 Oct 20 19:33 .
dr-xr-xr-x  6 root  wheel          512 Oct 20 15:33 ..
crw-r-----  1 root  operator    0, 163 Oct 20 19:33 500GB
crw-r-----  1 root  operator    0, 166 Oct 22 22:50 500GBa
crw-r-----  1 root  operator    0, 167 Oct 20 19:33 500GBc
crw-r-----  1 root  operator    0,  83 Oct 20 19:33 gm0
crw-r-----  1 root  operator    0,  85 Oct 20 19:33 gm0s1
crw-r-----  1 root  operator    0,  89 Oct 20 15:33 gm0s1a
crw-r-----  1 root  operator    0,  90 Oct 20 19:35 gm0s1b
crw-r-----  1 root  operator    0,  91 Oct 20 19:33 gm0s1c
crw-r-----  1 root  operator    0,  92 Oct 20 15:33 gm0s1d
crw-r-----  1 root  operator    0,  93 Oct 20 15:33 gm0s1e
crw-r-----  1 root  operator    0,  94 Oct 20 15:33 gm0s1f
crw-r-----  1 root  operator    0,  95 Oct 20 15:33 gm0s1g
[root@abyss /dev/mirror]#

The apparent problem is not that I can't get gmirror to setup the mirror with the 2 drives added. Or that I can mount the mirror in a location and it shows the full drive size.

But it does appear to be with the actual formatting of the drive and the slice creation. Like I mentioned earlier I was used to doing drive partitioning/slicing by sysinstall and I tried some newfs / bsdlabel stuff by hand on this one and need some baby-stepping to get this figured out. Oh when I did mount up the slice I thought I had created and it showed the full size in "df -h", I tried copying to it and it ran out of room and did a kernel panic. I *believe* it actually was a 24MB slice. Something is amiss here!

I do want both 500GB drives in a mirror with 1 partition using the full disk on both mounted at /500GB labeled 500GB in gmirror.

Any help would be awfully appreciated!
 
Something like this should do it:
Code: 
gmirror clear ad8
gmirror clear ad10
gmirror label -v -b load 500GB /dev/ad8
gmirror insert 500GB /dev/ad10
 
Right, but I'm not having a problem with the gmirror syntax or adding/removing devices. I'm having a drive slice problem.

By doing the commands you recommend I would still need to format the drives for use which leaves me in the same position?

Is the proper solution to delete the 500GB mirror entirely and then create a full-disk slice on ad8. Create a mirror with ad8 included and then add ad10 which would presumably copy the drive slice data over?
 
As far as I understood it you need to prepare ad8, set it up as you want it and then create the mirror.
 
So I am able to slice/label/mount each drive individually just fine outside of geom_mirror.

It's when I create a mirror that I run into problems.

I created one large partition on ad8 and ad10.
They were ad8s1d and ad10s1d respectively.
They mounted fine at /ad8 and /ad10.
I unmounted them and then began the gmirror process.

For the record I tried doing:
Code: 
gmirror clear ad8 (and yes I tried /dev/ad8)
gmirror clear ad10 (and yes I tried /dev/ad10)
But that reported errors saying it could not do so. The drives were not mounted when the gmirror clear cmd was issued.

Code: 
gmirror label -v -b load gm1 ad8 ad10

That created the mirror and added both.
Interesting this time that I didn't see any
synchronization going on. That's cool. It
must have noticed they were identical in setup.
Code: 
[root@abyss /dev/mirror]# gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ad4
                      ad6
mirror/gm1  COMPLETE  ad8
                      ad10
[root@abyss /dev/mirror]#

gmirror list provides the following for gm1:
Code: 
Geom name: gm1
State: COMPLETE
Components: 2
Balance: load
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 3798963086
Providers:
1. Name: mirror/gm1
   Mediasize: 500107861504 (466G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: ad8
   Mediasize: 500107862016 (466G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 3938878132
2. Name: ad10
   Mediasize: 500107862016 (466G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 1
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 4053620976


The part that *keeps* confusing me though is how it's doing the slice labeling for the dev references.
Code: 
[root@abyss /dev/mirror]# ls -lah
total 1
dr-xr-xr-x  2 root  wheel          512 Oct 20 19:33 .
dr-xr-xr-x  6 root  wheel          512 Oct 20 15:33 ..
crw-r-----  1 root  operator    0,  83 Oct 20 19:33 gm0
crw-r-----  1 root  operator    0,  85 Oct 20 19:33 gm0s1
crw-r-----  1 root  operator    0,  89 Oct 20 15:33 gm0s1a
crw-r-----  1 root  operator    0,  90 Oct 20 19:35 gm0s1b
crw-r-----  1 root  operator    0,  91 Oct 20 19:33 gm0s1c
crw-r-----  1 root  operator    0,  92 Oct 20 15:33 gm0s1d
crw-r-----  1 root  operator    0,  93 Oct 20 15:33 gm0s1e
crw-r-----  1 root  operator    0,  94 Oct 20 15:33 gm0s1f
crw-r-----  1 root  operator    0,  95 Oct 20 15:33 gm0s1g
crw-r-----  1 root  operator    0, 161 Oct 20 19:33 gm1
crw-r-----  1 root  operator    0, 165 Oct 20 19:33 gm1a
crw-r-----  1 root  operator    0, 166 Oct 20 19:33 gm1c
crw-r-----  1 root  operator    0, 173 Oct 20 19:33 gm1cs1
crw-r-----  1 root  operator    0, 183 Oct 20 19:33 gm1cs1c
crw-r-----  1 root  operator    0, 184 Oct 20 19:33 gm1cs1d
crw-r-----  1 root  operator    0, 168 Oct 20 19:33 gm1s1
crw-r-----  1 root  operator    0, 177 Oct 20 19:33 gm1s1c
crw-r-----  1 root  operator    0, 178 Oct 20 19:33 gm1s1d
[root@abyss /dev/mirror]#

Notice how gm0 is very simply laid out. It has the gm0 provider with the s1 sub and slices off of that. But if you look at gm1 it has a mess of stuff listed and none of it seems to make a lot of sense.

I'm still at a loss.
 
I suspect you want:

Code: 
bsdlabel -w /dev/mirror/gm1

Followed by:

Code: 
bsdlabel -e /dev/gm1

Then just edit the label and make partition "a" start at offset 16 and the full size of the disk (partition "c" size minus 16).

And finally:

Code: 
newfs -U /dev/mirror/gm1a
 
That appears to be what I want. I went in and checked my bsdlabel -e on gm1 and it already was setup how I wanted.

I guess the real confusion was which partition that it created was the right one.

Code: 
crw-r-----  1 root  operator    0, 161 Oct 20 19:33 gm1
crw-r-----  1 root  operator    0, 165 Oct 20 19:33 gm1a
crw-r-----  1 root  operator    0, 166 Oct 20 19:33 gm1c
crw-r-----  1 root  operator    0, 173 Oct 20 19:33 gm1cs1
crw-r-----  1 root  operator    0, 183 Oct 20 19:33 gm1cs1c
crw-r-----  1 root  operator    0, 184 Oct 20 19:33 gm1cs1d
crw-r-----  1 root  operator    0, 168 Oct 20 19:33 gm1s1
crw-r-----  1 root  operator    0, 177 Oct 20 19:33 gm1s1c
crw-r-----  1 root  operator    0, 178 Oct 20 19:33 gm1s1d

gm1a appears to be the one (as confirmed by bsdlabel -e gm1)
However, the other slices are confusing me. Is there a way to drop them from the dev/mirror? I'm trying to avoid rebooting as this machine is in production and is working beautifully.

I really appreciate the follow-up aragon. Thanks.
 
Unfortunately I'm not sure why those other device nodes are showing up, but if you haven't copied any data to the mirror yet, try this:

Code: 
dd if=/dev/zero of=/dev/mirror/gm1 bs=512 count=1024

After that, there should hopefully be only a gm1 device.

Then redo the steps from my last post.

If none of that works, I guess create a dedicated slice on gm1:

Code: 
fdisk -I /dev/mirror/gm1

Which should create gm1s1, and now create the bsdlabel on gm1s1.
 
About the /dev...
...s2 s3 s4 etc are "slices" (where one can bsdlabel,
newfs)
(I am not an expert but... probably...)
If they end in "a" they are already labels (I think)
If they end in "c" they are the whole disk...
If they end in "b" they are probably swap...
(From my failed attempts to get a gmirror install working,
which worked, but I could not figure out the "is it
there, is it here, what to do with what CLI, etc" and
eventually could not even destroy the gmirror with Fdisk,
used a Boot utility to wipe the s2 s3 or whatever...)
Went with another cutting-edge bsd subsystem which is
rocking so far...(not zfs, lowly hardware here.)
 
You must log in or register to reply here.
Back
Top